Browse the corpus

INTRODUCTION One of the main objectives of antenatal care is screening for fetal growth disturbances1. Although biomarkers in maternal blood have shown some potential for detecting fetal growth restriction2, 3, a recent systematic review suggested that none is sufficiently accurate to be recommended for use in clinical practice4. Clinicians, therefore, still rely routinely on clinical markers, including ultrasound measurements, to identify fetuses at risk5. Ultrasound evaluation of the fetus involves measuring head circumference (HC), abdominal circumference (AC) and femur length (FL), and the values can be combined to calculate an estimated fetal weight (EFW); this estimate is often used alone in clinical practice without considering the individual measurements. However, we believe that arguments concerning the most appropriate single parameter to use are inappropriate because clinicians should use all the tools available in their armamentarium for making crucial clinical decisions that have major implications for both mothers and newborns.

idering the individual measurements. However, we believe that arguments concerning the most appropriate single parameter to use are inappropriate because clinicians should use all the tools available in their armamentarium for making crucial clinical decisions that have major implications for both mothers and newborns. The development of international EFW standards is overdue, and these should share the same conceptual basis as the published INTERGROWTH‐21st standards for HC, AC and FL, size at birth and postnatal growth in preterm infants6, 7, 8, 9. These standards would perfectly complement the World Health Organization (WHO) Child Growth Standards10, thereby enabling continuity of assessment of human growth from early pregnancy to childhood11. Therefore, the objectives of this component of the INTERGROWTH‐21st Project were: (1) to develop a formula to estimate fetal weight based on ultrasound biometry and birth weight; and (2) to construct international EFW standards for fetuses at 22 to 40 weeks' gestation. SUBJECTS AND METHODS INTERGROWTH‐21st is an international, multicenter, population‐based project consisting of a number of components, including the Fetal Growth Longitudinal Study (FGLS) and INTERBIO‐21st Fetal Study (FS).

The development of international EFW standards is overdue, and these should share the same conceptual basis as the published INTERGROWTH‐21st standards for HC, AC and FL, size at birth and postnatal growth in preterm infants6, 7, 8, 9. These standards would perfectly complement the World Health Organization (WHO) Child Growth Standards10, thereby enabling continuity of assessment of human growth from early pregnancy to childhood11. Therefore, the objectives of this component of the INTERGROWTH‐21st Project were: (1) to develop a formula to estimate fetal weight based on ultrasound biometry and birth weight; and (2) to construct international EFW standards for fetuses at 22 to 40 weeks' gestation. SUBJECTS AND METHODS INTERGROWTH‐21st is an international, multicenter, population‐based project consisting of a number of components, including the Fetal Growth Longitudinal Study (FGLS) and INTERBIO‐21st Fetal Study (FS). FGLS was conducted between 27 April 2009 and 2 March 2014 in eight urban areas: the cities of Pelotas (Brazil), Turin (Italy), Muscat (Oman), Oxford (UK) and Seattle (USA); the Shunyi County of Beijing (China); the central area of Nagpur (India); and the Parklands suburb of Nairobi (Kenya). The primary aim was to study longitudinally the health and development of fetuses into infancy, by monitoring growth, health, nutrition and neurodevelopment from less than 14 weeks' gestation to 2 years of age, so as to produce prescriptive growth standards to complement the existing WHO Child Growth Standards. This was achieved by studying a cohort of healthy, well‐nourished, pregnant women who were at low risk of adverse maternal and perinatal outcomes at both population and individual levels. The study details have been described elsewhere9, 12.

iptive growth standards to complement the existing WHO Child Growth Standards. This was achieved by studying a cohort of healthy, well‐nourished, pregnant women who were at low risk of adverse maternal and perinatal outcomes at both population and individual levels. The study details have been described elsewhere9, 12. In contrast, FS recruited an unselected cohort of pregnant women, between 8 February 2012 and 24 December 2015, from three FGLS sites (Pelotas, Nairobi, Oxford), and three new sites (Aga Khan University Hospital, Karachi, Pakistan; Shoklo Malaria Research Unit, Mae Sot, Thailand; and Baragwanath Hospital, Soweto, South Africa). The primary aim was to study the effects of various intrauterine exposures (e.g. malnutrition, anemia, human immunodeficiency virus, malaria) on growth, health, nutrition, neurodevelopment and the epigenome, over the same developmental age range, i.e. from less than 14 weeks' gestation to 2 years of age. To develop the EFW formula requires as many pregnancies as possible that have a standardized scan and birth‐weight measurement. In order to achieve this we included fetuses from both FGLS and FS; only those that had an ultrasound scan within 14 days of birth were included in the calculations. To develop the international standards for EFW, the formula derived was then applied to the healthy FGLS population from which the International Fetal Growth Standards were produced6.

ncluded fetuses from both FGLS and FS; only those that had an ultrasound scan within 14 days of birth were included in the calculations. To develop the international standards for EFW, the formula derived was then applied to the healthy FGLS population from which the International Fetal Growth Standards were produced6. The INTERGROWTH‐21st Project was approved by the Oxfordshire Research Ethics Committee ‘C’ (reference: 08/H0606/139), the research ethics committees of the individual participating institutions and the corresponding regional health authorities in which the project was implemented. Participants provided written consent to be involved in the study. Standard procedures In both studies women were recruited at less than 14 weeks' gestation. All women underwent ultrasound measurement of fetal crown–rump length (CRL) using standardized methodology13, 14. In FGLS, gestational age was based on the date of the last menstrual period (LMP) provided it was certain, the woman had a regular 24–32‐day menstrual cycle and she had not been using hormonal contraception or breastfeeding in the preceding 2 months, and any discrepancy between the gestational ages based on LMP and CRL, between 9 + 0 and 13 + 6 weeks, was ≤ 7 days. In FS, gestational age was determined by CRL measurement alone, using the same formula loaded onto all study ultrasound machines15; if known, the date of the LMP was recorded.

in the preceding 2 months, and any discrepancy between the gestational ages based on LMP and CRL, between 9 + 0 and 13 + 6 weeks, was ≤ 7 days. In FS, gestational age was determined by CRL measurement alone, using the same formula loaded onto all study ultrasound machines15; if known, the date of the LMP was recorded. Following the dating scan, women were scanned every 5 weeks (±1 week), so that the possible ranges were 14–18, 19–23, 24–28, 29–33, 34–38 and 39–42 weeks' gestation. At each visit, fetal HC, biparietal diameter (BPD), occipitofrontal diameter (OFD), AC and FL were measured three times from three separately obtained ultrasound images of each structure. The detailed measurement protocols, including graphical displays of measurement techniques, and the unique standardization procedures for all measurements and sonographer training have been reported elsewhere13, 16. In addition, all documentation, protocols, quality‐control procedures, data collection forms and electronic transfer strategies are freely available on the INTERGROWTH‐21st website.

nt techniques, and the unique standardization procedures for all measurements and sonographer training have been reported elsewhere13, 16. In addition, all documentation, protocols, quality‐control procedures, data collection forms and electronic transfer strategies are freely available on the INTERGROWTH‐21st website. Briefly, head measurements were taken in an axial view at the level of the thalami, with an angle of insonation as close as possible to 90°. The head had to be oval in shape, symmetrical, centrally positioned and filling at least 30% of the monitor screen. The midline echo (representing the falx cerebri) had to be broken anteriorly, at a third of its length, by the cavum septi pellucidi. The thalami had to be located symmetrically on either side of the midline. Calipers were then placed on the outer border of the parietal bones (outer to outer) at the widest or longest part of the skull for the BPD and OFD, respectively; HC was measured using the ellipse facility on the outer border of the skull.

cidi. The thalami had to be located symmetrically on either side of the midline. Calipers were then placed on the outer border of the parietal bones (outer to outer) at the widest or longest part of the skull for the BPD and OFD, respectively; HC was measured using the ellipse facility on the outer border of the skull. AC measurements were taken in a cross‐sectional view of the fetal abdomen as close as possible to circular in shape, with the umbilical vein in the anterior third (at the level of the portal sinus), with the stomach bubble visible. The sonographer was instructed to avoid applying too much pressure with the transducer, which can distort the circular shape of the fetal abdomen. The abdomen had to fill at least 30% of the monitor screen, and the spine had to be at either the 3 or 9 o'clock position to avoid internal shadowing; the kidneys and bladder had not to be visible. For the measurements, the contour of the ellipse was placed on the outer border of the abdomen.

hape of the fetal abdomen. The abdomen had to fill at least 30% of the monitor screen, and the spine had to be at either the 3 or 9 o'clock position to avoid internal shadowing; the kidneys and bladder had not to be visible. For the measurements, the contour of the ellipse was placed on the outer border of the abdomen. Finally, FL was measured using a longitudinal view of the fetal thigh closest to the probe and with the femur as close as possible to the horizontal plane. The angle of insonation of the ultrasound beam was about 90°, with the full length of the bone visualized, unobscured by shadowing from adjacent bony parts, and the femur had to fill at least 30% of the monitor screen. The intersection of the calipers was placed on the outer borders of the edges of the femoral diaphysis (outer to outer) ensuring clear femoral edges; ultrasound artifacts of the femoral edges such as the proximal trochanter or pointed femoral spurs were not included in the measurement (detailed methods and a graphical display of how the bone structures are localized are available on the INTERGROWTH‐21st website).

physis (outer to outer) ensuring clear femoral edges; ultrasound artifacts of the femoral edges such as the proximal trochanter or pointed femoral spurs were not included in the measurement (detailed methods and a graphical display of how the bone structures are localized are available on the INTERGROWTH‐21st website). The same type of ultrasound machine, a Philips HD‐9 with curvilinear abdominal transducers C5‐2, C6‐3 and V7‐3 (Philips Ultrasound, Bothell, WA, USA), was used at all sites. To avoid expected value bias, the machine was adapted so that fetal measurements were not visible to the sonographer on the screen. Only after three measurements of each structure had been recorded were the average values revealed for clinical purposes. All ultrasound data were submitted electronically to the study database. Data were entered locally directly onto the web‐based system17. After taking each set of measurements, sonographers scored the quality of their images on the basis of standard image‐scoring criteria18, 19. Images that did not score the maximum number of points were repeated until the best possible score was achieved. The quality‐control methods used across all sites are described in detail elsewhere18, 20.

measurements, sonographers scored the quality of their images on the basis of standard image‐scoring criteria18, 19. Images that did not score the maximum number of points were repeated until the best possible score was achieved. The quality‐control methods used across all sites are described in detail elsewhere18, 20. Birth weight was measured within 12 h of birth using identical electronic scales (Seca, Hangzhou, China) at all sites. The equipment, which was calibrated twice a week, was selected for accuracy, precision and robustness, as shown previously21. Measurement procedures were standardized on the basis of WHO recommendations to ensure maximum validity and each measurement was collected independently by two study anthropometrists22, 23. If the difference between the two measurements exceeded the maximum allowable difference of 5 g, then both observers independently retook that measurement a second and, if necessary, a third time. The training, standardization, monitoring processes and quality‐control methods used across all sites are described in detail elsewhere22, 23. Statistical analysis Estimation of fetal weight

Birth weight was measured within 12 h of birth using identical electronic scales (Seca, Hangzhou, China) at all sites. The equipment, which was calibrated twice a week, was selected for accuracy, precision and robustness, as shown previously21. Measurement procedures were standardized on the basis of WHO recommendations to ensure maximum validity and each measurement was collected independently by two study anthropometrists22, 23. If the difference between the two measurements exceeded the maximum allowable difference of 5 g, then both observers independently retook that measurement a second and, if necessary, a third time. The training, standardization, monitoring processes and quality‐control methods used across all sites are described in detail elsewhere22, 23. Statistical analysis Estimation of fetal weight From the FGLS and FS cohorts, we identified all live babies without any congenital abnormality who were born at > 24 weeks' gestation and within 14 days of the last ultrasound scan. Given the study design, we expected the births to have occurred uniformly between 0 and 14 days after the last ultrasound scan, i.e. we expected there to be a mean time of 7–8 days between the last scan and birth. This cut‐off allowed a greater number of births at low gestational ages to be included, for which most of the existing formulae have been prone to prediction error, probably because scant data exist for estimation24. Potential predictors for birth weight were: HC, BPD, OFD, AC and FL, in cm or transformed into Z‐scores using the INTERGROWTH‐21st equations6;

f births at low gestational ages to be included, for which most of the existing formulae have been prone to prediction error, probably because scant data exist for estimation24. Potential predictors for birth weight were: HC, BPD, OFD, AC and FL, in cm or transformed into Z‐scores using the INTERGROWTH‐21st equations6; gestational age on the day of the last scan, in weeks; symphysis–fundus height, in cm; amniotic fluid, assessed by the deepest vertical pool and amniotic fluid index in cm; cross‐sectional head area and abdominal area computed from their orthogonal diameters, in cm2. We hypothesized that the contribution of any given anthropometric measurement to EFW might vary with gestational age. Therefore, we also considered interactions between HC, BPD, OFD, AC and FL and gestational age on the day of the last scan. Statistical modeling was conducted using second‐degree fractional polynomials25.

We hypothesized that the contribution of any given anthropometric measurement to EFW might vary with gestational age. Therefore, we also considered interactions between HC, BPD, OFD, AC and FL and gestational age on the day of the last scan. Statistical modeling was conducted using second‐degree fractional polynomials25. Some prediction bias would be expected because of significant growth between the day of the last scan and birth24, 26, 27. We addressed this issue by calculating the expected EFW on the day of the ultrasound scan, using the following steps: (1) in pregnancies from FGLS and FS delivering within 14 days from the last scan, we developed a model to predict birth weight from the most recent ultrasound measurements; (2) in the complete FGLS dataset, we calculated EFW from ultrasound biometry using the previous model and fitted a second‐degree fractional polynomial for mean weight as a function of gestational age between 22 and 40 weeks; (3) returning to the dataset of births within 14 days (step 1), we calculated, for each fetus, the expected weight at the time of the last scan by subtracting the average weight gain between the time of the last scan and birth using the model built in step 2; (4) this calculated weight was then used for further modeling.

) returning to the dataset of births within 14 days (step 1), we calculated, for each fetus, the expected weight at the time of the last scan by subtracting the average weight gain between the time of the last scan and birth using the model built in step 2; (4) this calculated weight was then used for further modeling. As expected, owing to the prospective, population‐based design of FGLS, most births occurred close to 40 weeks' gestation, meaning that the scatter of observations across the 22–40‐week window was very uneven. We were aware that estimation using the complete dataset would yield very accurate estimates at 40 weeks' gestation, where the greatest contribution of the data is found, but with limited model validity for lower birth weights. To overcome this problem and allow accurate birth‐weight estimation over the whole range of observed data, we constructed a bootstrap model selection and estimation procedure based on resampling of the complete dataset under an approximately uniform distribution of birth weight28, 29, 30, i.e. birth weight was divided into 500‐g strata and each sample was built by randomly selecting five observations with replacement from each stratum. In a first resampling run of 100 samples, candidate models, which include three elements (the variables, the coefficients and the respective fractional polynomial powers), were elicited using the backward elimination algorithm described by Ambler and Royston31, which provides protection against over‐fitting. In a second step, the coefficients of all candidate models were estimated in B = 1000 bootstrap samples: in each sample, a single model was selected using Akaike's Information Criterion. Candidate models were then ordered by their frequency of selection within the 1000 samples, and the five most frequent models were kept for further assessment of goodness of fit.

didate models were estimated in B = 1000 bootstrap samples: in each sample, a single model was selected using Akaike's Information Criterion. Candidate models were then ordered by their frequency of selection within the 1000 samples, and the five most frequent models were kept for further assessment of goodness of fit. Assessment of goodness of fit in the complete dataset relied on inspection of residuals with quantile–quantile (q‐q) plots and residuals vs fitted plots. Given that we estimated fetal weight at the time of the last scan using an average model for growth, we investigated the bias of our model for EFW by calculating the mean of percent prediction errors defined by the formula (100 × (EFW − birth weight)/birth weight), for decreasing time‐to‐birth intervals (i.e. from 14 to 0 days). Finally, we also calculated the absolute percent prediction error defined by the mean of the absolute prediction errors. Construction of reference centiles The construction of reference centiles was based solely on FGLS data. The sample size was based on pragmatic and statistical considerations; the latter focused on the precision and accuracy of one extreme centile, i.e. the 3rd or 97th centile, and regression‐based reference limits32, 33. We have shown that a sample of 4000 women would obtain a precision of 0.03 SD at the 3rd or 97th centile. Further details on the precision obtained at the 5th or 10th centile by sample size (ranging from 500–6000) have been included in a table in a previous publication34.

e, and regression‐based reference limits32, 33. We have shown that a sample of 4000 women would obtain a precision of 0.03 SD at the 3rd or 97th centile. Further details on the precision obtained at the 5th or 10th centile by sample size (ranging from 500–6000) have been included in a table in a previous publication34. The data from all the study sites were pooled to construct the Fetal Growth Standards6, 12, using the same statistical approach adopted by WHO in constructing their Child Growth Standards10. The statistical methods used were based on published recommendations complemented by recent scientific reviews5, 35, 36, 37, 38. Our overall aim was to produce centiles that change smoothly with age and maximize simplicity without compromising model fit.

ch adopted by WHO in constructing their Child Growth Standards10. The statistical methods used were based on published recommendations complemented by recent scientific reviews5, 35, 36, 37, 38. Our overall aim was to produce centiles that change smoothly with age and maximize simplicity without compromising model fit. We explored the following statistical methods: mean and SD method using fractional polynomials25; Cole's lambda (λ), mu (μ), and sigma (σ) (LMS) method39, 40, 41, which estimates three age‐specific parameters (the median (μ), coefficient of variation (σ), and a Box–Cox power transformation at each gestational age to remove skewness (λ), thereby making the data roughly normally distributed); the LMST method (λ, μ, σ, assuming Box–Cox t distribution), which assumes a shifted and scaled (truncated) t distribution to take account of skewness and leptokurtosis42; and the LMSP method (λ, μ, σ, assuming Box–Cox power exponential distribution), which assumes a Box–Cox power exponential distribution to take account of skewness, platykurtosis and leptokurtosis43. Furthermore, to present the curves, we assessed three smoothing techniques: fractional polynomials, cubic splines and penalized splines25, 44, 45. Using de‐trended q‐q plots (worm plots), significant evidence of deviations from normality was seen so we resorted to using the more complex LMS, LMST and LMSP methods allowing for skewness and kurtosis46.

We explored the following statistical methods: mean and SD method using fractional polynomials25; Cole's lambda (λ), mu (μ), and sigma (σ) (LMS) method39, 40, 41, which estimates three age‐specific parameters (the median (μ), coefficient of variation (σ), and a Box–Cox power transformation at each gestational age to remove skewness (λ), thereby making the data roughly normally distributed); the LMST method (λ, μ, σ, assuming Box–Cox t distribution), which assumes a shifted and scaled (truncated) t distribution to take account of skewness and leptokurtosis42; and the LMSP method (λ, μ, σ, assuming Box–Cox power exponential distribution), which assumes a Box–Cox power exponential distribution to take account of skewness, platykurtosis and leptokurtosis43. Furthermore, to present the curves, we assessed three smoothing techniques: fractional polynomials, cubic splines and penalized splines25, 44, 45. Using de‐trended q‐q plots (worm plots), significant evidence of deviations from normality was seen so we resorted to using the more complex LMS, LMST and LMSP methods allowing for skewness and kurtosis46. As most of the women had four to six ultrasound scans, the effect of correlated data within fetuses was investigated. First, in a sensitivity analysis, a random observation time was sampled for each fetus and the modeled centiles in this subset were compared visually with the complete dataset. The approach is justified by the experimental design of the study, which ensures non‐informative observation times47. This analysis showed minimal or no change in estimated centiles (median, 3rd and 97th centiles) over the whole 22–40 weeks' gestational‐age range. Second, we considered mixed‐effect models accounting for repeated measurements within LMS, LMST and LMSP frameworks. This analysis also showed no impact on the estimated centiles. The best fit was found using a three‐parameter Box–Cox Gaussian distribution (i.e. the LMS method) for the response variable with a second‐degree fractional polynomial functional form for gestational age. This method also gives estimated SDs of EFW, allowing estimation of centiles.

no impact on the estimated centiles. The best fit was found using a three‐parameter Box–Cox Gaussian distribution (i.e. the LMS method) for the response variable with a second‐degree fractional polynomial functional form for gestational age. This method also gives estimated SDs of EFW, allowing estimation of centiles. Goodness of fit for the overall model was assessed by comparing empirical centiles (calculated per completed week of gestation) with the fitted centiles, using de‐trended q‐q plots of the residuals across gestational age46, and plots of residuals vs fitted values. All analyses were carried out in R statistical software48 using the Generalised Additive Models for Location, Scale and Shape (GAMLSS) framework49, 50. RESULTS To create an EFW formula, the subsets of 2404 babies in the FGLS (n = 1556) and FS (n = 848) who were born within 14 days of the last ultrasound scan were examined (Table 1): 130 (5.4%) were born preterm (< 37 weeks' gestation) and 78 (3.2%) were born term with low birth weight (< 2500 g and ≥ 37 weeks' gestation). The mean time between the last ultrasound scan and birth was 7.7 (range, 0–14) days and was uniformly distributed, except for 0 days (i.e. birth on day of last scan), which occurred in only 34 (1.4%) cases. Table 1 Gestational age at birth and birth weight of a subset of babies in the INTERBIO‐21st Fetal Study (FS) and the Fetal Growth Longitudinal Study (FGLS) of the INTERGROWTH‐21st Project who were born within 14 days of last ultrasound scan

RESULTS To create an EFW formula, the subsets of 2404 babies in the FGLS (n = 1556) and FS (n = 848) who were born within 14 days of the last ultrasound scan were examined (Table 1): 130 (5.4%) were born preterm (< 37 weeks' gestation) and 78 (3.2%) were born term with low birth weight (< 2500 g and ≥ 37 weeks' gestation). The mean time between the last ultrasound scan and birth was 7.7 (range, 0–14) days and was uniformly distributed, except for 0 days (i.e. birth on day of last scan), which occurred in only 34 (1.4%) cases. Table 1 Gestational age at birth and birth weight of a subset of babies in the INTERBIO‐21st Fetal Study (FS) and the Fetal Growth Longitudinal Study (FGLS) of the INTERGROWTH‐21st Project who were born within 14 days of last ultrasound scan Parameter FS (n = 848) FGLS (n = 1556) Total (n = 2404) Gestational age at birth < 28 weeks 2 (0.2) 1 (0.1) 3 (0.1) 28–32 weeks 3 (0.4) 6 (0.4) 9 (0.4) 32–37 weeks 56 (6.6)  62 (4.0)  118 (4.9) 37–43 weeks 787 (92.8)  1487 (95.6)   2274 (94.6) Birth weight < 1000 g 1 (0.1) 2 (0.1) 3 (0.1) 1000–1499 g 5 (0.6) 1 (0.1) 6 (0.2) 1500–1999 g 9 (1.1) 15 (1.0)  24 (1.0) 2000–2499 g 79 (9.3)  76 (4.9)  155 (6.4) ≥ 2500 g 754 (88.9)  1462 (94.0)   2216 (92.2) Data are given as n (%).

weeks 56 (6.6)  62 (4.0)  118 (4.9) 37–43 weeks 787 (92.8)  1487 (95.6)   2274 (94.6) Birth weight < 1000 g 1 (0.1) 2 (0.1) 3 (0.1) 1000–1499 g 5 (0.6) 1 (0.1) 6 (0.2) 1500–1999 g 9 (1.1) 15 (1.0)  24 (1.0) 2000–2499 g 79 (9.3)  76 (4.9)  155 (6.4) ≥ 2500 g 754 (88.9)  1462 (94.0)   2216 (92.2) Data are given as n (%). Following correction for potential growth between the last scan and birth (steps 1–4 in the statistical methods), the actual fetal weight at the time of the last scan was best estimated as a function of AC and HC with the following formula: logEFW=5.084820−54.06633×AC/1003−95.80076×AC/1003×logAC/100+3.136370×HC/100 where EFW is expressed in g, AC and HC in cm, and the log function designates the natural logarithm. None of the other covariates including FL, BPD, OFD, gestational age, symphysis–fundus height, amniotic fluid indices or interactions between biometric measurements and gestational age was retained in the selection process. This model suggests a linear relationship between log(EFW) and HC. Despite the negative coefficients, the two terms involving AC describe an increasing sigmoid‐shaped relationship between AC and birth weight (Figure S1) for a fixed HC value of 26 cm (the average value at 28 weeks' gestation6). The relationship between birth weight and HC is plotted in Figure S2, for a fixed AC value of 23 cm (the average at 28 weeks' gestation6).

ms involving AC describe an increasing sigmoid‐shaped relationship between AC and birth weight (Figure S1) for a fixed HC value of 26 cm (the average value at 28 weeks' gestation6). The relationship between birth weight and HC is plotted in Figure S2, for a fixed AC value of 23 cm (the average at 28 weeks' gestation6). The performance of the formula for EFW was assessed both by mean and absolute percent prediction errors; mean percent prediction error is used as a measure of potential bias of EFW due to growth between the last scan and birth, while mean absolute prediction error represents the dispersion of the errors. The mean percent prediction error steadily tended towards zero as the time interval between the last scan and birth decreased. Prediction error was −10.7% (95% CI, −12.1 to −9.4%) in babies born exactly 14 days after the last scan (n = 196) and −0.8% (95% CI, −2.3 to 0.6%) in those born within 1 day (n = 198) (Figure S3), showing that our model was unbiased for predicting weight at the time of the last scan and that the correction we applied to compensate for time to birth was appropriate. In the group born within 1 day of the last scan, the mean absolute prediction error was 7.6%, with 80%, 90% and 95% of predicted weights falling within 11%, 14% and 18% of the true birth weight, respectively.

t the time of the last scan and that the correction we applied to compensate for time to birth was appropriate. In the group born within 1 day of the last scan, the mean absolute prediction error was 7.6%, with 80%, 90% and 95% of predicted weights falling within 11%, 14% and 18% of the true birth weight, respectively. Creation of the international EFW standards was based on the complete FGLS dataset. The gestational age‐specific observed and smoothed centiles for EFW are presented in Figure 1. Similarities between smoothed centile curves (3rd, 50th and 97th centiles) and observed values, assessed by gestational age‐specific comparisons, demonstrated excellent agreement. The overall differences between empirical and smoothed centiles were small, with mean ± SD differences of 16 ± 28 g, 13 ± 17 g and 5 ± 33 g for the 3rd, 50th and 97th centiles, respectively. Figure 1 Empirical () and smoothed () 3rd, 50th and 97th centiles for estimated fetal weight between 22 and 40 weeks' gestation. UOG-17347-FIG-0001-bThe 3rd, 10th, 50th, 90th and 97th fitted centile curves for EFW according to gestational age, which represent the international standards, are presented in Figure 2. The corresponding equations for λ(t), μ(t) and σ(t), are presented in Table 2, allowing readers to calculate Z‐scores. By estimating the EFW and knowing the gestational age, desired centiles can be calculated. For example, if AC = 26 cm and HC = 29 cm, at 30 + 0 weeks: logEFW=5.084820−54.06633×26/1003−95.80076×26/1003×log26/100+3.136370×29/100=7.312292. Therefore, EFW = exp(7.312292) = 1499 g.

2, allowing readers to calculate Z‐scores. By estimating the EFW and knowing the gestational age, desired centiles can be calculated. For example, if AC = 26 cm and HC = 29 cm, at 30 + 0 weeks: logEFW=5.084820−54.06633×26/1003−95.80076×26/1003×log26/100+3.136370×29/100=7.312292. Therefore, EFW = exp(7.312292) = 1499 g. Figure 2 Smoothed 3rd, 10th, 50th, 90th and 97th centile curves for estimated fetal weight. UOG-17347-FIG-0002-bTable 2 Equations for parameters and computation of Z‐scores and centiles for estimated fetal weight (EFW) in relation to gestational age (GA) in exact weeks Parameter Equation Skewness λ(GA) = − 4.257629 − 2162.234 × GA− 2 + 0.0002301829 × GA3 Mean μ(GA) = 4.956737 + 0.0005019687 × GA3 − 0.0001227065 × GA3 × log(GA) Coefficient of variation σ(GA) = 10− 4 × (− 6.997171 + 0.057559 × GA3 − 0.01493946 × GA3 × log(GA)) Z‐score Y = log(EFW) If λ(GA) = 0, Z(GA) = σ(GA)− 1 × log[Y/μ(GA)] If λ(GA) ≠ 0, Z(GA) = [σ(GA) × λ(GA)]− 1 × [(Y/μ(GA))λ(GA) − 1] Centiles Zαdefined by Pr(z ≤ Zα) = α for z ∼ N(0, 1), i. e. Zα = Φ− 1(α) If λ(GA) = 0, log[Cα(GA)] = μ(GA) × exp[σ(GA) × Zα] If λ(GA) ≠ 0, log[Cα(GA)] = μ(GA) × [Zα × σ(GA) × λ(GA) + 1]1/λ(GA) To compute the corresponding Z‐score at 30 weeks' gestation, using the equations in Table 2 we must first calculate: λ30=–4.257629−2162.234×30–2+0.0002301829×303=–0.4451729. µ30=4.956737+0.0005019687×303–0.0001227065×303×log30=7.241468. σ30=10–4×(–6.997171+0.057559×303–0.01493946×303×log30)=0.017517.

Z‐score Y = log(EFW) If λ(GA) = 0, Z(GA) = σ(GA)− 1 × log[Y/μ(GA)] If λ(GA) ≠ 0, Z(GA) = [σ(GA) × λ(GA)]− 1 × [(Y/μ(GA))λ(GA) − 1] Centiles Zαdefined by Pr(z ≤ Zα) = α for z ∼ N(0, 1), i. e. Zα = Φ− 1(α) If λ(GA) = 0, log[Cα(GA)] = μ(GA) × exp[σ(GA) × Zα] If λ(GA) ≠ 0, log[Cα(GA)] = μ(GA) × [Zα × σ(GA) × λ(GA) + 1]1/λ(GA) To compute the corresponding Z‐score at 30 weeks' gestation, using the equations in Table 2 we must first calculate: λ30=–4.257629−2162.234×30–2+0.0002301829×303=–0.4451729. µ30=4.956737+0.0005019687×303–0.0001227065×303×log30=7.241468. σ30=10–4×(–6.997171+0.057559×303–0.01493946×303×log30)=0.017517. Finally, Z=0.017517×–0.4451729–1×(7.312292/7.241468–0.4451729–1)=0.5617023. Similarly, the 3rd centile (α = 0.03), i.e. Z = −1.88 at 30 + 0 weeks, is calculated as follows using the equations in Table 2: logC0.0330=7.241468×–1.88×0.017517×–0.4451729+1)–1/0.4451729=7.008552. The 3rd centile for EFW at 30 weeks' gestation is therefore: C0.03(30) = exp(7.008552) = 1106 g. The actual values for the 3rd, 10th, 50th, 90th and 97th centiles according to gestational age are presented in Table S1.

Finally, Z=0.017517×–0.4451729–1×(7.312292/7.241468–0.4451729–1)=0.5617023. Similarly, the 3rd centile (α = 0.03), i.e. Z = −1.88 at 30 + 0 weeks, is calculated as follows using the equations in Table 2: logC0.0330=7.241468×–1.88×0.017517×–0.4451729+1)–1/0.4451729=7.008552. The 3rd centile for EFW at 30 weeks' gestation is therefore: C0.03(30) = exp(7.008552) = 1106 g. The actual values for the 3rd, 10th, 50th, 90th and 97th centiles according to gestational age are presented in Table S1. DISCUSSION The INTERGROWTH‐21st Project provides standards for early human growth based on populations that conform to the prescriptive approach recommended by the WHO21, 51. By prescriptive, we mean that we observed a cohort of prospectively enrolled women whose risk of adverse maternal and perinatal outcomes (including fetal growth restriction) was low, based on their individual clinical profiles and the socioeconomic and demographic characteristics of the underlying geographically diverse populations. In fact, the INTERGROWTH‐21st Project is unique because it has produced, for the first time, fetal ultrasound, newborn size and preterm postnatal growth datasets that have all been collected from the same underlying populations using the same rigorously applied methodologies.

erlying geographically diverse populations. In fact, the INTERGROWTH‐21st Project is unique because it has produced, for the first time, fetal ultrasound, newborn size and preterm postnatal growth datasets that have all been collected from the same underlying populations using the same rigorously applied methodologies. We now present international EFW standards to complement the existing set, along with a formula for EFW based on HC and AC. Compared with several previous formulae24, we found that FL did not improve the EFW, which agrees with previous work, in particular in growth‐restricted fetuses52. Furthermore, it is likely that incorporating FL into the formula would increase the prediction error, as its measurement is associated with the highest inter‐ and intraobserver variability compared with AC and HC53. Unusually, we lowered the starting gestational age to 22 weeks, 2 weeks below the customary cut‐off of 24 weeks' gestation for viability, for two reasons: to facilitate early recognition of fetal growth restriction around the recommended time of the second‐trimester anatomy scan and to anticipate a possible extension of the limit of viability54, 55.

ing gestational age to 22 weeks, 2 weeks below the customary cut‐off of 24 weeks' gestation for viability, for two reasons: to facilitate early recognition of fetal growth restriction around the recommended time of the second‐trimester anatomy scan and to anticipate a possible extension of the limit of viability54, 55. At the upper end of gestation, the centiles closely match those of the INTERGROWTH‐21st Newborn Size Standards at 40 weeks7. The 3rd, 50th and 97th EFW centiles at 40 weeks are 2554 g, 3338 g and 4121 g, respectively (Table S1), whereas for newborns (sexes combined) they are 2591 g, 3321 g and 4154 g, respectively (Figure S4). These similarities between fetal‐ and birth‐weight centiles suggest that our model is valid for developing a formula for EFW using ultrasound biometry. In contrast, there are significant discrepancies earlier in pregnancy (Figure 3). For example, at 33 weeks' gestation, the 3rd, 50th and 97th EFW centiles are 1495 g, 1954 g and 2529 g, respectively (Table S1); for newborns (sexes combined), they are 1190 g, 1903 g and 2715 g, respectively. It is possible that these differences are due to an overrepresentation of small, as well as, to a lesser extent, large babies in preterm births, even in the selected pregnant and newborn populations we studied. Figure 3 Comparison of fitted 3rd, 50th and 97th centiles for estimated fetal weight () with those of INTERGROWTH‐21st preterm postnatal weight, with both sexes combined ().

In contrast, there are significant discrepancies earlier in pregnancy (Figure 3). For example, at 33 weeks' gestation, the 3rd, 50th and 97th EFW centiles are 1495 g, 1954 g and 2529 g, respectively (Table S1); for newborns (sexes combined), they are 1190 g, 1903 g and 2715 g, respectively. It is possible that these differences are due to an overrepresentation of small, as well as, to a lesser extent, large babies in preterm births, even in the selected pregnant and newborn populations we studied. Figure 3 Comparison of fitted 3rd, 50th and 97th centiles for estimated fetal weight () with those of INTERGROWTH‐21st preterm postnatal weight, with both sexes combined (). UOG-17347-FIG-0003-bThe EFW formula and standards we present are also unique because we avoided the many common limitations identified by previous reviews5, 24: retrospective design; use of routinely obtained measurements; suboptimal pregnancy dating strategies; variable time‐to‐birth without controlling for bias; absence of prospective ultrasound quality control, standardization and calibration of equipment; hospital‐based sampling; absence of sampling from a healthy, well‐nourished, underlying population; and no blinding of measurements.

boptimal pregnancy dating strategies; variable time‐to‐birth without controlling for bias; absence of prospective ultrasound quality control, standardization and calibration of equipment; hospital‐based sampling; absence of sampling from a healthy, well‐nourished, underlying population; and no blinding of measurements. Conversely, our standards are prescriptive, whereas reference charts describe only fetal size at a given place and time. The standards were derived prospectively, population‐based and multinational. We have shown (using several analytical strategies) that the eight populations were consistently similar and could be pooled to create international standards51. Uniform research methods, protocols, processes and measurement tools were used throughout; these were combined with standardized identical equipment, training, a centralized electronic data management system and close monitoring of staff. The analytical approach aimed at identifying and correcting potential biases, and followed WHO recommendations to present the observed and smoothed data and explore the best fitting model with an a‐priori strategy56.

cal equipment, training, a centralized electronic data management system and close monitoring of staff. The analytical approach aimed at identifying and correcting potential biases, and followed WHO recommendations to present the observed and smoothed data and explore the best fitting model with an a‐priori strategy56. Using ultrasound, we examined separately HC, AC and FL, providing a comprehensive evaluation of structures that have different growth patterns; these measurements are often combined to calculate EFW. There are advantages in using a summary approximation: it is the most commonly measured marker of size at birth; as birth weight is associated with morbidity and mortality, it is helpful when counseling parents and enables pediatricians to make management decisions57; it may also help to refine the management of large babies.

re advantages in using a summary approximation: it is the most commonly measured marker of size at birth; as birth weight is associated with morbidity and mortality, it is helpful when counseling parents and enables pediatricians to make management decisions57; it may also help to refine the management of large babies. However, there are also disadvantages in using only a single summary measure of size: first, there is a loss of the most granular information available when using the individual measurements, in terms of fetal skeletal and fat‐based growth. Second, the fact that the individual measurement errors are compounded means that estimation is prone to inaccuracy; previous studies have shown that 95% prediction intervals for random error are in the region of ± 14% of birth weight, and this is a particular problem in low‐ and high‐birth‐weight babies24. Finally, as for other ultrasound measurements, there are numerous locally‐derived EFW equations and reference charts24 but, until now, no international standards existed, unlike the situation for newborn size and infant growth7, 8, 10. This may be, at least partly, responsible for the poor efficiency of screening strategies using biometry and EFW58.

d measurements, there are numerous locally‐derived EFW equations and reference charts24 but, until now, no international standards existed, unlike the situation for newborn size and infant growth7, 8, 10. This may be, at least partly, responsible for the poor efficiency of screening strategies using biometry and EFW58. Therefore, we strongly recommend that, for clinical use, all individual fetal measurements, together with the summary measure of EFW, should be used together to make clinical decisions. In perinatal medicine, there is no room for a quick, minimalist approach that might lead to the early delivery of an at‐risk fetus. Finally, implementation of the standards may raise concerns regarding the generalizability of data originating from a limited number of sites and/or a highly selected, low‐risk population. As we have argued previously11, having separate standards for a given country, institution or ethnic group has no biological basis and makes little sense in modern, multicultural societies. The international INTERGROWTH‐21st standards describe optimal growth and can be used to assess both individuals and populations. Supporting information Appendix S1 Members of the International Fetal and Newborn Growth Consortium for the 21st Century (INTERGROWTH‐21st and INTERBIO‐21st) and its Committees Click here for additional data file. Figure S1 Relationship between fetal weight and abdominal circumference in the final model, plotted for a fixed head circumference of 26 cm. Click here for additional data file.

Appendix S1 Members of the International Fetal and Newborn Growth Consortium for the 21st Century (INTERGROWTH‐21st and INTERBIO‐21st) and its Committees Click here for additional data file. Figure S1 Relationship between fetal weight and abdominal circumference in the final model, plotted for a fixed head circumference of 26 cm. Click here for additional data file. Figure S2 Relationship between fetal weight and head circumference in the final model, plotted for a fixed abdominal circumference of 23 cm. Click here for additional data file. Figure S3 Bias in estimation of fetal weight as a function of time to birth, showing mean percent prediction error and 95% CI according to time between last ultrasound scan and birth. Click here for additional data file. Figure S4 Gestational age‐specific centiles for estimated fetal weight (blue) and birth weight (red). 3rd, 10th, 50th, 90th and 97th centiles are shown. Click here for additional data file. Table S1 Estimated fetal weight per completed week of gestation at 3rd, 10th, 50th, 90th and 97th centiles Click here for additional data file.

Figure S4 Gestational age‐specific centiles for estimated fetal weight (blue) and birth weight (red). 3rd, 10th, 50th, 90th and 97th centiles are shown. Click here for additional data file. Table S1 Estimated fetal weight per completed week of gestation at 3rd, 10th, 50th, 90th and 97th centiles Click here for additional data file. ACKNOWLEDGMENTS This project was supported by a generous grant from the Bill & Melinda Gates Foundation to the University of Oxford, for which we are very grateful. We would also like to thank the Health Authorities in Pelotas, Brazil; Beijing, China; Nagpur, India; Turin, Italy; Nairobi, Kenya; Kilifi, Kenya; Muscat, Oman; Karachi, Pakistan; Johannesburg, South Africa; Mae Sot, Thailand; Oxford, UK and Seattle, USA, who facilitated the project by allowing participation of these study sites as collaborating centers. We are extremely grateful to Philips Medical Systems, who provided the ultrasound equipment and technical assistance throughout the project. We also thank MedSciNet UK Ltd for setting up the INTERGROWTH‐21st and INTERBIO‐21st websites and for the development, maintenance and support of the online data management system. We are indebted to GAPPS for the supply of sample processing kits for INTERBIO‐21st.

pment and technical assistance throughout the project. We also thank MedSciNet UK Ltd for setting up the INTERGROWTH‐21st and INTERBIO‐21st websites and for the development, maintenance and support of the online data management system. We are indebted to GAPPS for the supply of sample processing kits for INTERBIO‐21st. We thank the parents and infants who participated in the studies and the more than 200 members of the research teams who made the implementation of this project possible. The participating hospitals included: Brazil, Pelotas (Hospital Miguel Piltcher, Hospital São Francisco de Paula, Santa Casa de Misericórdia de Pelotas, and Hospital Escola da Universidade Federal de Pelotas); China, Beijing (Beijing Obstetrics & Gynecology Hospital, Shunyi Maternal & Child Health Centre, and Shunyi General Hospital); India, Nagpur (Ketkar Hospital, Avanti Institute of Cardiology Private Limited, Avantika Hospital, Gurukrupa Maternity Hospital, Mulik Hospital & Research Centre, Nandlok Hospital, Om Women's Hospital, Renuka Hospital & Maternity Home, Saboo Hospital, Brajmonhan Taori Memorial Hospital, and Somani Nursing Home); Italy, Turin (Ospedale Infantile Regina Margherita Sant' Anna and Azienda Ospedaliera Ordine Mauriziano); Kenya, Nairobi (Aga Khan University Hospital, MP Shah Hospital and Avenue Hospital); Kenya, Kilifi, (The Kilifi District Hospital); Oman, Muscat (Khoula Hospital, Royal Hospital, Wattayah Obstetrics & Gynaecology Poly Clinic, Wattayah Health Centre, Ruwi Health Centre, Al‐Ghoubra Health Centre and Al‐Khuwair Health Centre); Pakistan, Karachi (Aga Khan Hospital); South Africa, Johannesburg (Chris Hani Baragwanath Academic Hospital); Thailand, Mae Sot (Maela , Wang Pha, and Mawker Thai Clinics); UK, Oxford (John Radcliffe Hospital) and USA, Seattle (University of Washington Hospital, Swedish Hospital, and Providence Everett Hospital). Members of the INTERGROWTH‐21st and its committees are listed in Appendix S1. Full acknowledgement of all those who contributed to the development of the INTERGROWTH‐21st Project protocol appears at www.intergrowth21.org.uk and www.interbio21.org.uk.

INTRODUCTION Infants who are born small‐for‐gestational age (SGA) are at increased risk of short‐term neonatal morbidity and mortality1, 2, 3, and longer‐term complications extending into adult life, including cardiovascular disease and Type‐II diabetes mellitus4. SGA is commonly defined as birth weight under a centile threshold. For infants under the 10th centile of the population, this group includes constitutionally small infants and those with fetal growth restriction, the latter defined as failure of a fetus to reach its full growth potential. Use of birth‐weight centiles customized for additional maternal (height, weight, ethnicity, parity) and fetal (sex) variables improves identification of those fetuses at risk of adverse perinatal outcome, including stillbirth and neonatal death5.

latter defined as failure of a fetus to reach its full growth potential. Use of birth‐weight centiles customized for additional maternal (height, weight, ethnicity, parity) and fetal (sex) variables improves identification of those fetuses at risk of adverse perinatal outcome, including stillbirth and neonatal death5. The underlying pathophysiology of fetal growth restriction is complex, but poor placentation plays a key role in a substantial proportion of SGA, particularly in women with preterm hypertensive disorders and when associated with adverse perinatal outcome. There is a need for a test in the second half of pregnancy to identify those at highest risk of delivering a SGA infant. Markers of placental function could offer a useful adjunct to current ultrasonographic techniques to improve risk stratification, enabling identification of those at greatest risk and minimizing unnecessary interventions in lower‐risk women. Several biomarkers have been suggested as potential predictors of fetal growth restriction, but to date none has been shown to have adequate accuracy to support incorporation into clinical practice6. The fetuses of women with suspected hypertensive disorders of pregnancy who present before 37 weeks' gestation are at increased risk of fetal growth restriction, but the optimal strategy for identifying such fetuses remains unclear.

been shown to have adequate accuracy to support incorporation into clinical practice6. The fetuses of women with suspected hypertensive disorders of pregnancy who present before 37 weeks' gestation are at increased risk of fetal growth restriction, but the optimal strategy for identifying such fetuses remains unclear. As part of a large prospective study in women presenting with suspected pre‐eclampsia (PE), we sought first to evaluate 47 biomarkers (identified by an extensive literature search) and then compare the best performing biomarker(s) against currently utilized ultrasound parameters for determining subsequent delivery of a SGA infant and adverse perinatal outcome. METHODS The PELICAN study was a prospective observational study, undertaken between January 2011 and February 2012 in seven consultant‐led maternity units in the UK and Ireland. The role of placental growth factor (PlGF) in determining the need for delivery within 14 days of sampling for PE in this study has been reported previously7, and this was a planned further analysis.

study, undertaken between January 2011 and February 2012 in seven consultant‐led maternity units in the UK and Ireland. The role of placental growth factor (PlGF) in determining the need for delivery within 14 days of sampling for PE in this study has been reported previously7, and this was a planned further analysis. Participants Study eligibility required the presence of signs or symptoms of suspected PE in women presenting between 20 + 0 and 36 + 6 weeks' gestation with a singleton pregnancy and aged ≥ 16 years; women with confirmed PE at enrollment were excluded. Written informed consent was obtained and baseline demographic and pregnancy‐specific data were entered into the study database. Blood was drawn into ethylenediamine‐tetraacetic acid at study enrollment and samples were centrifuged at 3000 rpm for 10 min. Plasma was extracted and stored at –80°C until analysis. Management of the women in the study followed the usual care pathways for women with suspected PE, as advised in the UK National Institute for Health and Care Excellence ‘Hypertension in Pregnancy’ guidelines8, with ultrasound assessment being undertaken as clinically indicated.

cted and stored at –80°C until analysis. Management of the women in the study followed the usual care pathways for women with suspected PE, as advised in the UK National Institute for Health and Care Excellence ‘Hypertension in Pregnancy’ guidelines8, with ultrasound assessment being undertaken as clinically indicated. Ultrasound assessments were undertaken by trained ultrasonographers at each study site as clinically indicated, using a variety of machines and following local protocols for the measurement of fetal biometry, amniotic fluid index and umbilical artery Doppler flow‐velocity waveforms (as occurred in clinical practice at the time of the study). Quality control was undertaken through local procedures rather than by the research team centrally. Estimated fetal weight (EFW) was calculated at each site using the Hadlock formula9. Additional parameters, including uterine artery, fetal middle cerebral artery and ductus venosus Doppler studies, were not universally reported and therefore their performance could not be compared with biomarker performance. As study sites were reporting abnormal ultrasound assessment using a variety of parameters (including abdominal circumference (AC) and EFW < 10th, < 5th and < 3rd centiles), the most commonly reported parameter, AC or EFW < 10th centile, was chosen to enable comparison across sites. The presence of AC or EFW < 10th centile, oligohydramnios (amniotic fluid index < 5th centile) or absent/reversed end‐diastolic flow was recorded by participating midwives.

FW < 10th, < 5th and < 3rd centiles), the most commonly reported parameter, AC or EFW < 10th centile, was chosen to enable comparison across sites. The presence of AC or EFW < 10th centile, oligohydramnios (amniotic fluid index < 5th centile) or absent/reversed end‐diastolic flow was recorded by participating midwives. Final diagnoses of maternal hypertensive disorders of pregnancy were assigned, following agreement by an adjudication panel of experts, using definitions from the American College of Obstetricians and Gynecologists practice bulletin10. SGA was defined as birth weight < 3rd customized centile (SGA‐3), with birth weight < 10th customized centile (SGA‐10) being a secondary outcome, calculated using the Gestation‐Related Optimal Weight (GROW) method by freely available software11. All diagnoses were assigned without knowledge of any biomarker values.

10. SGA was defined as birth weight < 3rd customized centile (SGA‐3), with birth weight < 10th customized centile (SGA‐10) being a secondary outcome, calculated using the Gestation‐Related Optimal Weight (GROW) method by freely available software11. All diagnoses were assigned without knowledge of any biomarker values. The prespecified first part of the biomarker analysis presented here relates to two groups of women in predefined gestational age strata enrolled with a singleton pregnancy and suspected preterm PE: Group 1 at 20 + 0 to 34 + 6 weeks' gestation and Group 2 at 35 + 0 to 36 + 6 weeks' gestation. For comparison with ultrasound parameters, the second part of the analysis was restricted to women with an ultrasound scan performed within 14 days of blood sampling at enrollment. The principal prespecified outcome of both analyses was delivery of a SGA‐3 infant3. The prespecified secondary outcome measures were birth weight < 10th customized centile (SGA‐10) and adverse perinatal outcome. Adverse perinatal outcome was predefined as the presence of any of the following complications: antepartum/intrapartum fetal or neonatal death, intraventricular hemorrhage, periventricular leukomalacia, seizure, retinopathy of prematurity, respiratory distress syndrome, bronchopulmonary dysplasia, necrotizing enterocolitis or admission to neonatal unit for > 48 h at term. Adverse maternal outcome was defined as the presence of any of the following: maternal death, eclampsia, stroke, cortical blindness or retinal detachment, hypertensive encephalopathy, systolic blood pressure ≥ 160 mmHg, myocardial infarction, intubation (other than for Cesarean section), pulmonary edema, platelets < 50 × 109/L (without transfusion), disseminated intravascular coagulation, thrombotic thrombocytopenic purpura/hemolytic uremic syndrome, hepatic dysfunction (alanine transaminase ≥ 70 IU/L), hepatic hematoma or rupture, acute fatty liver of pregnancy, creatinine > 150 μmol/L, renal dialysis, placental abruption, major postpartum hemorrhage or major infection.

nated intravascular coagulation, thrombotic thrombocytopenic purpura/hemolytic uremic syndrome, hepatic dysfunction (alanine transaminase ≥ 70 IU/L), hepatic hematoma or rupture, acute fatty liver of pregnancy, creatinine > 150 μmol/L, renal dialysis, placental abruption, major postpartum hemorrhage or major infection. Biomarker measurement An initial panel of biomarkers was selected based on a-priori knowledge of an association with PE, a biological role in placentation or a role in cellular mechanisms involved in the pathogenesis of PE, e.g. angiogenesis, inflammation, coagulation. The full list of 47 biomarkers, measured with 57 assays (in which potentially biologically important assays of different epitope specificity were available) was generated following a review of the literature, appraisal of selected bibliography and consultation with medical experts (Table S1). Samples were labeled and transported to the laboratory, where they were spun at 3000 rpm for 10 min. Plasma samples were tested for PlGF using the Triage PlGF Test (Alere Inc., San Diego, CA, USA) by trained laboratory staff at the study site where the sample was taken (as previously published). The additional 56 biomarker assays were analyzed in a central laboratory facility (Alere, San Diego, CA, USA) and full details of assay methods are given in Appendix S1 and Table S2. All participants were delivered and pregnancy outcomes recorded before biomarker concentrations were analyzed and revealed, and all laboratory staff were blinded to clinical outcomes.

ed in a central laboratory facility (Alere, San Diego, CA, USA) and full details of assay methods are given in Appendix S1 and Table S2. All participants were delivered and pregnancy outcomes recorded before biomarker concentrations were analyzed and revealed, and all laboratory staff were blinded to clinical outcomes. Statistical analysis Standard distributional checks showed high levels of skewness for all 57 assays, which were consistent with underlying values of log normal distribution. Logged values of these biomarkers were therefore used. Before considering the pregnancy outcomes, statistical factor analysis of biomarker data was undertaken, reducing the 47 biomarkers into a smaller number of highly correlated groups, solely on the basis of the correlations between the biomarkers. Factor summary scores were then calculated for all the women. Consideration of scree plots and eigenvalues (> 2) identified the most important factors for further analysis12. These factors were rotated (orthogonal varimax method) so that each factor related strongly (correlation > 0.6) to a small number of biomarkers only (factor analysis displayed in Table S3).

or all the women. Consideration of scree plots and eigenvalues (> 2) identified the most important factors for further analysis12. These factors were rotated (orthogonal varimax method) so that each factor related strongly (correlation > 0.6) to a small number of biomarkers only (factor analysis displayed in Table S3). The factor scores were entered into a multiple logistic regression model for prediction of subsequent SGA. Two factors (and their biomarkers), with significant odds ratios for prediction of SGA < 3rd centile, were identified for further investigation (Tables S4 and S5). Stepwise logistic regression was used to determine which biomarkers appeared to provide additional information beyond that derived from PlGF, and prediction scores were extracted for the best combinations. A comparison of areas under the receiver–operating characteristics curves (AUCs) of individual biomarkers and combinations was made to see if any of the additional information was both consistent and clinically useful. Significance was assessed through the use of a non‐parametric test, which allowed for non‐independence of observations on the same participant, with Bonferroni correction for multiple testing. Some biomarkers, with high uniqueness scores, were not strongly associated with any factor. To investigate whether any of these biomarkers had prognostic power in addition to that provided by PlGF and biomarkers identified earlier, stepwise logistic regression analysis was undertaken.

The factor scores were entered into a multiple logistic regression model for prediction of subsequent SGA. Two factors (and their biomarkers), with significant odds ratios for prediction of SGA < 3rd centile, were identified for further investigation (Tables S4 and S5). Stepwise logistic regression was used to determine which biomarkers appeared to provide additional information beyond that derived from PlGF, and prediction scores were extracted for the best combinations. A comparison of areas under the receiver–operating characteristics curves (AUCs) of individual biomarkers and combinations was made to see if any of the additional information was both consistent and clinically useful. Significance was assessed through the use of a non‐parametric test, which allowed for non‐independence of observations on the same participant, with Bonferroni correction for multiple testing. Some biomarkers, with high uniqueness scores, were not strongly associated with any factor. To investigate whether any of these biomarkers had prognostic power in addition to that provided by PlGF and biomarkers identified earlier, stepwise logistic regression analysis was undertaken. The best performing biomarker was then assessed using standard test performance indices (sensitivity, specificity, predictive values and AUC) against currently utilized ultrasound parameters in the subgroup of women with an ultrasound scan within 14 days of blood sampling, for the prediction of SGA and adverse perinatal outcome. A sensitivity analysis was conducted excluding those fetuses in which the scan on the day of enrollment had abnormal findings (AC or EFW < 10th centile, oligohydramnios or absent/reversed end diastolic flow (n = 20)).

asound scan within 14 days of blood sampling, for the prediction of SGA and adverse perinatal outcome. A sensitivity analysis was conducted excluding those fetuses in which the scan on the day of enrollment had abnormal findings (AC or EFW < 10th centile, oligohydramnios or absent/reversed end diastolic flow (n = 20)). Statistical analysis was carried out in the statistical package Stata version 11.2 (College Station, TX, USA); statistical significance was taken as P < 0.05. The prespecified sample size was calculated for accurate estimation of the sensitivity (within 10%) and specificity (within 6%) of a biomarker, assuming a sensitivity of 0.90, specificity of 0.90 and two‐tailed 95% CIs, for determining the primary endpoint; this required 62 patients with PE and 150 women not meeting the primary endpoint. The study is reported in accordance with Strengthening the Reporting of Observational Studies in Epidemiology guidelines (Table S6)13, it was approved by East London Research Ethics Committee (ref. 10/H0701/117) and it followed institutional guidelines. RESULTS Between January 2011 and February 2012, 274 women presenting with suspected PE and a singleton pregnancy were enrolled between 20 + 0 and 34 + 6 weeks' gestation (Group 1), and 123 women were enrolled between 35 + 0 and 36 + 6 weeks (Group 2) (Figure 1). Figure 1 Flowchart of study participants. BW, birth weight; PlGF, placental growth factor; SGA, small‐for‐gestational age.

RESULTS Between January 2011 and February 2012, 274 women presenting with suspected PE and a singleton pregnancy were enrolled between 20 + 0 and 34 + 6 weeks' gestation (Group 1), and 123 women were enrolled between 35 + 0 and 36 + 6 weeks (Group 2) (Figure 1). Figure 1 Flowchart of study participants. BW, birth weight; PlGF, placental growth factor; SGA, small‐for‐gestational age. UOG-17490-FIG-0001-bThe characteristics of the women in Group 1 at booking and enrollment are shown in Table 1 and details of maternal and neonatal outcomes are given in Table 2. Of the 274 women in Group 1, 96 (35%) delivered a SGA infant < 3rd centile (SGA‐3) (of whom 90% developed PE) and 130 (47%) delivered a SGA infant < 10th centile (of whom 81% developed PE). Adverse perinatal outcome was three times higher (39% vs 13%) in cases complicated by SGA‐3 than in those delivering an infant with appropriate‐for‐gestational‐age birth weight. Stillbirth occurred in six pregnancies, in five of which the birth weight was < 3rd centile. In all stillbirth cases the PlGF concentration was < 5th centile at enrollment, and predated the detection of ultrasound abnormalities by 7 to 39 days and the occurrence of stillbirth by 10 to 53 days. Table 1 Characteristics of 274 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia (PE), according to subsequent birth weight of infant

UOG-17490-FIG-0001-bThe characteristics of the women in Group 1 at booking and enrollment are shown in Table 1 and details of maternal and neonatal outcomes are given in Table 2. Of the 274 women in Group 1, 96 (35%) delivered a SGA infant < 3rd centile (SGA‐3) (of whom 90% developed PE) and 130 (47%) delivered a SGA infant < 10th centile (of whom 81% developed PE). Adverse perinatal outcome was three times higher (39% vs 13%) in cases complicated by SGA‐3 than in those delivering an infant with appropriate‐for‐gestational‐age birth weight. Stillbirth occurred in six pregnancies, in five of which the birth weight was < 3rd centile. In all stillbirth cases the PlGF concentration was < 5th centile at enrollment, and predated the detection of ultrasound abnormalities by 7 to 39 days and the occurrence of stillbirth by 10 to 53 days. Table 1 Characteristics of 274 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia (PE), according to subsequent birth weight of infant Characteristic Women with SGA infant < 3rd centile (n = 96) Women with SGA infant < 10th centile (n = 130) Women with infant ≥ 10th centile (n = 144) At booking Age (years) 31.9 (27.2–36.2) 31.9 (27.4–36.4) 31.7 (26.3–35.6) BMI (kg/m2) 26.8 (24.1–31.2) 28.0 (23.9–32.8) 29.3 (24.7–34.9) White ethnicity 63 (66) 87 (67) 92 (64) Highest systolic BP (mmHg) 120 (110–130) 121 (110–130) 120 (110–130) Highest diastolic BP (mmHg) 74 (65–81) 74 (65–81) 75 (68–82) Smoker at booking 17 (18) 24 (18) 29 (20) Quit smoking during pregnancy 10 (10) 14 (11) 19 (13) Previous PE requiring delivery < 34 weeks 15 (16) 18 (14) 12 (8) Chronic hypertension 11 (11) 21 (16) 23 (16) At enrollment GA at sampling (weeks) 31.0 (27.6–33.0) 31.0 (27.6–33.1) 31.1 (28.0–33.6) New‐onset hypertension 60 (63) 80 (62) 65 (45) Worsening of underlying hypertension 16 (17) 24 (18) 32 (22) New‐onset dipstick proteinuria 58 (60) 79 (61) 71 (49) Highest systolic BP (mmHg) 147 (137–160) 148 (138–160) 141 (128–156) Highest diastolic BP (mmHg) 94 (83–100) 94 (83–100) 90 (80–100) Values given as median (interquartile range) or n (%).

80 (62) 65 (45) Worsening of underlying hypertension 16 (17) 24 (18) 32 (22) New‐onset dipstick proteinuria 58 (60) 79 (61) 71 (49) Highest systolic BP (mmHg) 147 (137–160) 148 (138–160) 141 (128–156) Highest diastolic BP (mmHg) 94 (83–100) 94 (83–100) 90 (80–100) Values given as median (interquartile range) or n (%). BMI, body mass index; BP, blood pressure; GA, gestational age; SGA, small‐for‐gestational age. Table 2 Delivery characteristics and maternal and neonatal outcome of 274 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia (PE), according to subsequent birth weight of infant Characteristic Women with SGA infant < 3rd centile (n = 96) Women with SGA infant < 10th centile (n = 130) Women with infant ≥ 10th centile (n = 144) Onset of labor* Spontaneous 3 (3) 7 (5) 32 (22) Induced 29 (30) 42 (32) 64 (44) Prelabor Cesarean section 64 (67) 80 (62) 46 (32) Mode of delivery*

Table 2 Delivery characteristics and maternal and neonatal outcome of 274 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia (PE), according to subsequent birth weight of infant Characteristic Women with SGA infant < 3rd centile (n = 96) Women with SGA infant < 10th centile (n = 130) Women with infant ≥ 10th centile (n = 144) Onset of labor* Spontaneous 3 (3) 7 (5) 32 (22) Induced 29 (30) 42 (32) 64 (44) Prelabor Cesarean section 64 (67) 80 (62) 46 (32) Mode of delivery* Spontaneous vaginal 15 (16) 25 (19) 45 (31) Assisted vaginal 5 (5) 8 (6) 21 (15) Cesarean section 75 (78) 95 (73) 78 (54) Adverse maternal outcome† 44 (46) 61 (47) 56 (39) GA at delivery (weeks) 33.8 (30.8–36.1) 34.4 (31.4–37.3) 38.1 (36.0–39.4) Fetal death 5 (5) 5 (4) 1 (1) Neonatal death 2 (2) 2 (2) 0 (0) Birth weight (g) 1537 (1043–1910) 1660 (1200–2310) 3128 (2698–3545) SGA < 10th birth‐weight centile 96 (100) 130 (100) 0 (0) SGA < 3rd birth‐weight centile 96 (100) 96 (74) 0 (0) SGA < 1st birth‐weight centile 68 (71) 68 (52) 0 (0) Adverse perinatal outcome† 37 (39) 41 (32) 19 (13) Maternal diagnosis No disease 0 (0) 1 (1) 21 (15) Gestational hypertension 1 (1) 1 (1) 25 (17) Chronic hypertension 4 (4) 12 (9) 16 (11) PE 86 (90) 106 (82) 59 (41) HELLP syndrome 1 (1) 1 (1) 1 (1) Other 4 (4) 9 (7) 22 (15) Values given as median (interquartile range) or n (%). * Data missing for onset of labor in one woman and mode of delivery in two. † Adverse outcome defined in main text. GA, gestational age; SGA, small‐for‐gestational age.

Spontaneous vaginal 15 (16) 25 (19) 45 (31) Assisted vaginal 5 (5) 8 (6) 21 (15) Cesarean section 75 (78) 95 (73) 78 (54) Adverse maternal outcome† 44 (46) 61 (47) 56 (39) GA at delivery (weeks) 33.8 (30.8–36.1) 34.4 (31.4–37.3) 38.1 (36.0–39.4) Fetal death 5 (5) 5 (4) 1 (1) Neonatal death 2 (2) 2 (2) 0 (0) Birth weight (g) 1537 (1043–1910) 1660 (1200–2310) 3128 (2698–3545) SGA < 10th birth‐weight centile 96 (100) 130 (100) 0 (0) SGA < 3rd birth‐weight centile 96 (100) 96 (74) 0 (0) SGA < 1st birth‐weight centile 68 (71) 68 (52) 0 (0) Adverse perinatal outcome† 37 (39) 41 (32) 19 (13) Maternal diagnosis No disease 0 (0) 1 (1) 21 (15) Gestational hypertension 1 (1) 1 (1) 25 (17) Chronic hypertension 4 (4) 12 (9) 16 (11) PE 86 (90) 106 (82) 59 (41) HELLP syndrome 1 (1) 1 (1) 1 (1) Other 4 (4) 9 (7) 22 (15) Values given as median (interquartile range) or n (%). * Data missing for onset of labor in one woman and mode of delivery in two. † Adverse outcome defined in main text. GA, gestational age; SGA, small‐for‐gestational age. The predictive performance of the most promising biomarkers as depicted by AUCs is shown in Table 3; AUCs for all 47 biomarkers measured are given in Table S7, and individual median biomarker concentrations in women sampled prior to 35 weeks are shown in Table S8. In isolation, PlGF had the best predictive performance for the detection of SGA‐3 when measured before 35 weeks' gestation, with an AUC of 0.83 (sensitivity, 89.7% (95% CI, 81.7–94.9%); specificity, 58.7% (95% CI, 51.1–66.0%); positive predictive value, 53.8% (95% CI, 45.7–61.7%); and negative predictive value (NPV), 91.3% (95% CI, 84.6–95.8%)). Combinations of the most promising biomarkers (Table 3) showed only minimal non‐significant increases in AUC for the prediction of SGA‐3 (from 0.83 to 0.84) and SGA‐10 (from 0.79 to 0.80).

.1–66.0%); positive predictive value, 53.8% (95% CI, 45.7–61.7%); and negative predictive value (NPV), 91.3% (95% CI, 84.6–95.8%)). Combinations of the most promising biomarkers (Table 3) showed only minimal non‐significant increases in AUC for the prediction of SGA‐3 (from 0.83 to 0.84) and SGA‐10 (from 0.79 to 0.80). Table 3 Performance of individual biomarkers and their combinations (derived from logistic regression analysis) for prediction of small‐for‐gestational‐age (SGA) < 3rd centile and < 10th centile in 274 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia (PE) Biomarker AUC (95% CI) P * SGA < 3rd centile SGA < 10th centile Nephrin 0.63 (0.56–0.70) 0.62 (0.55–0.69) < 0.001 CPA‐4† 0.63 (0.57–0.70) 0.62 (0.55–0.68) < 0.001 sFlt‐1 0.73 (0.67–0.79) 0.69 (0.63–0.76) < 0.001 Endoglin 0.74 (0.68–0.80) 0.73 (0.67–0.79) < 0.001 PlGF† 0.83 (0.78–0.88) 0.79 (0.73–0.84) — PlGF/s‐Flt ratio† 0.80 (0.75–0.85) 0.77 (0.71–0.82)  0.004 PlGF/endoglin ratio† 0.82 (0.77–0.86) 0.78 (0.73–0.83)  0.204 PlGF† + CPA‐4† 0.83 (0.78–0.88) 0.79 (0.74–0.84)  0.560 PlGF† + nephrin 0.84 (0.79–0.88) 0.80 (0.74–0.85)  0.475 PlGF† + nephrin + CPA‐4† 0.84 (0.79–0.89) 0.80 (0.74–0.85)  0.390 * Comparison of performance of single biomarker (or combination) vs placental growth factor (PlGF) alone. † Low concentration of biomarker or low ratio correlates to disease. AUC, area under receiver–operating characteristics curve; CPA‐4, carboxypeptidase A4 sFlt‐1, soluble fms‐like tyrosine kinase‐1.

SGA < 3rd centile SGA < 10th centile Nephrin 0.63 (0.56–0.70) 0.62 (0.55–0.69) < 0.001 CPA‐4† 0.63 (0.57–0.70) 0.62 (0.55–0.68) < 0.001 sFlt‐1 0.73 (0.67–0.79) 0.69 (0.63–0.76) < 0.001 Endoglin 0.74 (0.68–0.80) 0.73 (0.67–0.79) < 0.001 PlGF† 0.83 (0.78–0.88) 0.79 (0.73–0.84) — PlGF/s‐Flt ratio† 0.80 (0.75–0.85) 0.77 (0.71–0.82)  0.004 PlGF/endoglin ratio† 0.82 (0.77–0.86) 0.78 (0.73–0.83)  0.204 PlGF† + CPA‐4† 0.83 (0.78–0.88) 0.79 (0.74–0.84)  0.560 PlGF† + nephrin 0.84 (0.79–0.88) 0.80 (0.74–0.85)  0.475 PlGF† + nephrin + CPA‐4† 0.84 (0.79–0.89) 0.80 (0.74–0.85)  0.390 * Comparison of performance of single biomarker (or combination) vs placental growth factor (PlGF) alone. † Low concentration of biomarker or low ratio correlates to disease. AUC, area under receiver–operating characteristics curve; CPA‐4, carboxypeptidase A4 sFlt‐1, soluble fms‐like tyrosine kinase‐1. Of women enrolled before 35 weeks, 129 had an ultrasound scan with all parameters recorded within 14 days of enrollment. The test performances of ultrasound parameters and PlGF (the best performing biomarker) for determining SGA‐3 and SGA‐10 are shown in Table 4 and Table S9, respectively, with PlGF alone having a higher sensitivity (SGA‐3, 93% (95% CI, 84–98%)) and NPV (SGA‐3, 90% (95% CI, 76–97%)) than any other indicator. While the addition of PlGF to currently used ultrasound parameters (AC or EFW < 10th centile) increased the sensitivity for the detection of SGA‐3 (from 71% to 97%), the addition of ultrasound parameters to PlGF measurement did not markedly enhance sensitivity (from 93% to 97%). Adverse perinatal outcome (excluding SGA in this definition) occurred in 22% (60 of 274 infants). In predicting composite adverse perinatal outcome, PlGF had the highest sensitivity (90%) and NPV (90%) compared with all ultrasound measurements (n = 129; Table 5). In a sensitivity analysis, performance of the ultrasound and PlGF variables was similar when those with an abnormal scan on the day of enrollment were excluded from the analysis (Tables S10 and S11).

come, PlGF had the highest sensitivity (90%) and NPV (90%) compared with all ultrasound measurements (n = 129; Table 5). In a sensitivity analysis, performance of the ultrasound and PlGF variables was similar when those with an abnormal scan on the day of enrollment were excluded from the analysis (Tables S10 and S11). Table 4 Performance of individual indicators and their combinations for prediction of small‐for‐gestational‐age (SGA) < 3rd customized birth‐weight centile in 129 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrollment

come, PlGF had the highest sensitivity (90%) and NPV (90%) compared with all ultrasound measurements (n = 129; Table 5). In a sensitivity analysis, performance of the ultrasound and PlGF variables was similar when those with an abnormal scan on the day of enrollment were excluded from the analysis (Tables S10 and S11). Table 4 Performance of individual indicators and their combinations for prediction of small‐for‐gestational‐age (SGA) < 3rd customized birth‐weight centile in 129 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrollment Indicator Sensitivity (95% CI) (%) Specificity (95% CI) (%) PPV (95% CI) (%) NPV (95% CI) (%) LR+ (95% CI) LR− (95% CI) Indicator in isolation AC or EFW < 10th centile 71.2 (57.9–82.2) 92.5 (83.4–97.5) 89.4 (76.9–96.5) 78.5 (67.8–86.9) 9.5 (4.0–22.5) 0.31 (0.21–0.47) Oligohydramnios* 18.6 (9.7–30.9) 98.5 (92.0–100.0) 91.7 (61.5–99.8) 57.9 (48.3–67.1) 12.5 (1.7–93.9) 0.83 (0.73–0.94) AREDF 20.3 (11.0–32.8) 98.5 (92.0–100.0) 92.3 (64.0–99.8) 58.4 (48.8–67.6) 13.6 (1.8–101.7) 0.81 (0.71–0.92) PlGF < 100 pg/mL 93.2 (83.5–98.1) 52.2 (39.7–64.6) 63.2 (52.2–73.3) 89.7 (75.8–97.1) 2.0 (1.5–2.5) 0.13 (0.05–0.34) Combinations of indicators AC or EFW < 10th centileor oligohydramnios or AREDF 72.9 (59.7–83.6) 91.0 (81.5–96.6) 87.8 (75.2–95.4) 79.2 (68.5–87.6) 8.1 (3.7–17.7) 0.30 (0.19–0.46) AC or EFW < 10th centile or PlGF < 100 pg/mL 96.6 (88.3–99.6) 49.3 (36.8–61.8) 62.6 (51.9–72.6) 94.3 (80.8–99.3) 1.9 (1.5–2.3) 0.07 (0.02–0.28) * Oligohydramnios defined as amniotic fluid index < 5th centile for gestational age.

.7–83.6) 91.0 (81.5–96.6) 87.8 (75.2–95.4) 79.2 (68.5–87.6) 8.1 (3.7–17.7) 0.30 (0.19–0.46) AC or EFW < 10th centile or PlGF < 100 pg/mL 96.6 (88.3–99.6) 49.3 (36.8–61.8) 62.6 (51.9–72.6) 94.3 (80.8–99.3) 1.9 (1.5–2.3) 0.07 (0.02–0.28) * Oligohydramnios defined as amniotic fluid index < 5th centile for gestational age. AC, abdominal circumference; AREDF, absent or reversed end‐diastolic flow in umbilical artery; EFW, estimated fetal weight; LR+/LR−, positive/negative likelihood ratio; NPV, negative predictive value; PlGF, placental growth factor; PPV, positive predictive value. Table 5 Performance of individual indicators and their combinations for prediction of adverse perinatal outcome in 129 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrollment

AC, abdominal circumference; AREDF, absent or reversed end‐diastolic flow in umbilical artery; EFW, estimated fetal weight; LR+/LR−, positive/negative likelihood ratio; NPV, negative predictive value; PlGF, placental growth factor; PPV, positive predictive value. Table 5 Performance of individual indicators and their combinations for prediction of adverse perinatal outcome in 129 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrollment Indicator Sensitivity (95% CI) (%) Specificity (95% CI) (%) PPV (95% CI) (%) NPV (95% CI) (%) LR+ (95% CI) LR− (95% CI) Indicator in isolation AC or EFW < 10th centile 48.7 (32.4–65.2) 67.8 (56.9–77.4) 40.4 (26.4–55.7) 74.7 (63.6–83.8) 1.5 (1.0–2.4) 0.76 (0.54–1.06) Oligohydramnios* 12.8 (4.3–27.4) 92.0 (84.1–96.7) 41.7 (15.2–72.3) 70.2 (60.9–78.4) 1.6 (0.5–4.7) 0.95 (0.83–1.09) AREDF 12.8 (4.3–27.4) 90.8 (82.7–95.9) 38.5 (13.9–68.4) 69.9 (60.6–78.2) 1.4 (0.5–4.0) 0.96 (0.84–1.10) PlGF < 100 pg/mL 89.7 (75.8–97.1) 40.2 (29.9–51.3) 40.2 (29.9–51.3) 89.7 (75.8–97.1) 1.5 (1.2–1.8) 0.25 (0.10–0.67) Combinations of indicators AC or EFW < 10th centile or oligohydramnios or AREDF 53.8 (37.2–69.9) 67.8 (56.9–77.4) 42.9 (28.8–57.8) 76.6 (65.6–85.5) 1.7 (1.1–2.6) 0.68 (0.47–0.98) AC or EFW < 10th centile or PlGF < 100 pg/mL 92.3 (79.1–98.4) 36.8 (26.7–47.8) 39.6 (29.5–50.4) 91.4 (76.9–98.2) 1.5 (1.2–1.8) 0.21 (0.07–0.64) * Oligohydramnios defined as amniotic fluid index < 5th centile for gestational age.

7.2–69.9) 67.8 (56.9–77.4) 42.9 (28.8–57.8) 76.6 (65.6–85.5) 1.7 (1.1–2.6) 0.68 (0.47–0.98) AC or EFW < 10th centile or PlGF < 100 pg/mL 92.3 (79.1–98.4) 36.8 (26.7–47.8) 39.6 (29.5–50.4) 91.4 (76.9–98.2) 1.5 (1.2–1.8) 0.21 (0.07–0.64) * Oligohydramnios defined as amniotic fluid index < 5th centile for gestational age. AC, abdominal circumference, AREDF, absent or reversed end‐diastolic flow in umbilical artery; EFW, estimated fetal weight; LR+/LR−, positive/negative likelihood ratio; NPV, negative predictive value; PlGF, placental growth factor; PPV, positive predictive value. A total of 123 women were enrolled between 35 + 0 and 36 + 6 weeks' gestation (Group 2); characteristics of these women at booking and enrollment and details of maternal and neonatal outcomes are described in Tables S12 and S13. AUCs for all 47 biomarkers measured between 35 + 0 and 36 + 6 weeks are given in Table S14. When measured in isolation, PlGF had an AUC of 0.69 for predicting SGA‐3 and 0.74 for SGA‐10; addition of carboxypeptidase A4 raised this to 0.77 for SGA‐3 and 0.81 for SGA‐10 (Table S15). The addition of other biomarkers yielded little benefit. In this group, PlGF had a higher sensitivity than all other currently used ultrasound indicators in predicting SGA infants (Tables S16 and S17) and adverse perinatal outcomes (Table S18).

peptidase A4 raised this to 0.77 for SGA‐3 and 0.81 for SGA‐10 (Table S15). The addition of other biomarkers yielded little benefit. In this group, PlGF had a higher sensitivity than all other currently used ultrasound indicators in predicting SGA infants (Tables S16 and S17) and adverse perinatal outcomes (Table S18). DISCUSSION Our study shows that PlGF measurement has a high sensitivity and NPV in the determination of subsequent delivery of a SGA infant, and in the prediction of adverse perinatal outcome, in women presenting with suspected preterm PE. We evaluated SGA < 3rd birth‐weight centile to identify fetuses more likely to be growth restricted, rather than constitutionally small. Our study suggests that PlGF measurement has a potential role alongside ultrasound assessment in the surveillance of high‐risk women with suspected PE and that integration of PlGF with current ultrasound parameters may increase detection rates of SGA. Ultrasound has an essential role in the detection of falling growth velocity, oligohydramnios and abnormal umbilical artery Doppler waveforms, which will continue to be used to stratify surveillance and time delivery appropriately. The use of PlGF for the prediction of SGA relates to this high‐risk group of women with suspected PE and cannot be generalized to low‐risk healthy pregnant women14.

, oligohydramnios and abnormal umbilical artery Doppler waveforms, which will continue to be used to stratify surveillance and time delivery appropriately. The use of PlGF for the prediction of SGA relates to this high‐risk group of women with suspected PE and cannot be generalized to low‐risk healthy pregnant women14. Of 46 additional biomarkers evaluated in isolation or in combination with PlGF, there was minimal added value to the predictive performance of PlGF alone, and these markers are unlikely to be of utility in the clinical setting. It is possible that serial PlGF concentrations, with measurements made closer to outcome, may further improve the predictive ability while other biomarkers may only become significant closer to outcome. Placental pathology would have been a useful additional tool for assessing fetal growth restriction, but it was not available for this study. A possible source of intervention bias is that ultrasound results were revealed to clinicians while biomarker results were not. At the time of the study in the UK, it was not common practice to deliver for falling growth velocity alone (i.e. pre‐empting delivery of a SGA infant) unless the EFW fell below the < 10th centile. Adverse perinatal outcome (excluding SGA) was chosen as a secondary outcome to evaluate the performance of the variables on this additional clinically meaningful endpoint.

mmon practice to deliver for falling growth velocity alone (i.e. pre‐empting delivery of a SGA infant) unless the EFW fell below the < 10th centile. Adverse perinatal outcome (excluding SGA) was chosen as a secondary outcome to evaluate the performance of the variables on this additional clinically meaningful endpoint. This study enrolled women who presented for obstetric assessment with a broad range of symptoms and signs of suspected PE, including those with underlying maternal disease. This is more informative than evaluating the tests against normal healthy pregnant women (as in a case–control study), as it is likely to reflect more closely the test performance in the usual clinical setting. The multicenter nature of the study, incorporating women of geographic and ethnic diversity, adds to the generalizability of the results. Further strengths of the study are that all final clinical diagnoses were adjudicated by a panel of medical experts and all clinical and laboratory staff were unaware of biomarker results until completion of the study.

incorporating women of geographic and ethnic diversity, adds to the generalizability of the results. Further strengths of the study are that all final clinical diagnoses were adjudicated by a panel of medical experts and all clinical and laboratory staff were unaware of biomarker results until completion of the study. It is a feature of our study that the assessments (including ultrasound examination) were performed in a local healthcare setting without referral, ultrasound or management protocols being dictated centrally by the research team. It is a strength that this pragmatic approach makes it likely that the prognostic variables would have comparable performance when translated beyond the research setting, with the findings directly generalizable to similar healthcare settings. However, it is a potential limitation that such an approach does not reflect assessment of ultrasound as undertaken in some healthcare systems (e.g. by a maternal–fetal medicine subspecialist).

formance when translated beyond the research setting, with the findings directly generalizable to similar healthcare settings. However, it is a potential limitation that such an approach does not reflect assessment of ultrasound as undertaken in some healthcare systems (e.g. by a maternal–fetal medicine subspecialist). The findings of this study relate to similar healthcare settings in which same‐day ultrasound assessment is not routinely undertaken for women presenting with suspected PE, owing to national guideline recommendations or lack of trained ultrasonographers. In settings in which all women with suspected PE undergo same‐day ultrasound assessment by a maternal–fetal medicine subspecialist, the performance of ultrasound may be different. As we included scans performed within 14 days after blood sampling, ultrasound scans may have been undertaken closer to the clinical endpoint (and would therefore not have been expected to introduce bias against ultrasound test performance).

–fetal medicine subspecialist, the performance of ultrasound may be different. As we included scans performed within 14 days after blood sampling, ultrasound scans may have been undertaken closer to the clinical endpoint (and would therefore not have been expected to introduce bias against ultrasound test performance). We are not aware of any study that has compared such a wide panel of biomarkers (n = 47) for the prediction of subsequent SGA in women with suspected PE. Reports on the capability of PlGF to predict SGA have been conflicting. Initial small case–control studies in the first and second trimesters for the prediction of SGA found no significant relationship15, 16, but subsequent larger case–control studies and several prospective cohorts measuring PlGF in the second and first trimesters have reported an association between low PlGF concentrations and early‐onset pre‐eclampsia17, 18, stillbirth19 and SGA20, 21, 22. The few small (n = 21 or fewer), mainly case–control studies in which measurement was undertaken in the third trimester (including at time of delivery), generally concur with our finding of low PlGF concentration in women with subsequent delivery of a SGA infant23, 24, 25, 26, particularly those with significant underlying placental pathology27. As impaired placental function underpins a substantial proportion of cases of SGA (and PE)28, an angiogenic placental factor such as PlGF has biological plausibility for prediction. A recent systematic review of 53 studies (principally of first‐ and second‐trimester prediction, and with no studies of PlGF in a similar cohort to this study) investigated the value of biomarkers in the prediction of fetal growth restriction in singleton pregnancies and concluded that PlGF emerged as the most promising of the 37 biomarkers reported6. The finding that PlGF measurements also predict adverse perinatal outcome is supported by two other studies29, 30, but the first evaluated PlGF measurements in the first trimester while the second reported a combined maternal and perinatal adverse outcome.

F emerged as the most promising of the 37 biomarkers reported6. The finding that PlGF measurements also predict adverse perinatal outcome is supported by two other studies29, 30, but the first evaluated PlGF measurements in the first trimester while the second reported a combined maternal and perinatal adverse outcome. SGA has the highest population‐attributable risk value (23%) for stillbirth of all pregnancy‐specific disorders31. In this study cohort, five of six cases complicated by stillbirth delivered an infant with a birth weight < 3rd centile. In a setting in which ultrasound is not routinely performed in all women with suspected PE, PlGF measurement might facilitate earlier and more accurate detection of SGA associated with perinatal mortality, allowing appropriate surveillance of those at highest risk with the aim of improving outcome. Such a strategy could allow appropriate targeting of resources to at‐risk pregnancies with subsequent improvements in maternal and fetal outcomes. Supporting information Appendix S1 Biomarker assays. Table S1 List of biomarker abbreviations and units Table S2 Biomarker assay information Table S3 Results of factor analysis: loadings of biomarkers on five largest factors (eigenvalues > 2) after varimax rotation showing loadings > 0.6 only and uniqueness > 0.6

SGA has the highest population‐attributable risk value (23%) for stillbirth of all pregnancy‐specific disorders31. In this study cohort, five of six cases complicated by stillbirth delivered an infant with a birth weight < 3rd centile. In a setting in which ultrasound is not routinely performed in all women with suspected PE, PlGF measurement might facilitate earlier and more accurate detection of SGA associated with perinatal mortality, allowing appropriate surveillance of those at highest risk with the aim of improving outcome. Such a strategy could allow appropriate targeting of resources to at‐risk pregnancies with subsequent improvements in maternal and fetal outcomes. Supporting information Appendix S1 Biomarker assays. Table S1 List of biomarker abbreviations and units Table S2 Biomarker assay information Table S3 Results of factor analysis: loadings of biomarkers on five largest factors (eigenvalues > 2) after varimax rotation showing loadings > 0.6 only and uniqueness > 0.6 Table S4 Odds ratios derived from multiple logistic regression analysis of the five factors for prediction of delivery of SGA infant in women presenting with suspected pre‐eclampsia before 35 weeks' gestation (odds ratios are for a change of 1 SD in the factor score). Factors 3 and 4 (with significant odds ratios for prediction of SGA infant < 3rd centile) were taken forward for further analysis Table S5 Odds ratios derived from multiple logistic regression analysis of five factors for prediction of delivery of SGA infant in women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia

Table S4 Odds ratios derived from multiple logistic regression analysis of the five factors for prediction of delivery of SGA infant in women presenting with suspected pre‐eclampsia before 35 weeks' gestation (odds ratios are for a change of 1 SD in the factor score). Factors 3 and 4 (with significant odds ratios for prediction of SGA infant < 3rd centile) were taken forward for further analysis Table S5 Odds ratios derived from multiple logistic regression analysis of five factors for prediction of delivery of SGA infant in women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia Table S6 STROBE checklist Table S7 ROC curve areas (with 95% CI) for individual biomarkers to predict small‐for‐gestational age (SGA) < 3rd and < 10th customized birth‐weight centiles in women presenting with suspected pre‐eclampsia before 35 weeks' gestation Table S8 Individual median biomarker concentrations (quartiles) in women presenting before 35 weeks' gestation with suspected pre‐eclampsia Table S9 Predictive performance of individual indicators and their combinations, to predict delivery of small‐for‐gestational age (SGA) < 10th customized birth‐weight centile in 129 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment

redictive performance of individual indicators and their combinations, to predict delivery of small‐for‐gestational age (SGA) < 10th customized birth‐weight centile in 129 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment Table S10 Predictive performance of individual indicators and their combinations, to predict delivery of small‐for‐gestational age (SGA) < 3rd customized birth‐weight centile in 109 women presenting at 20 + 0 to 34 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment, excluding those with known abnormal scan findings on day of enrolment Table S11 Test performance statistics for individual indicators and their combinations to predict adverse perinatal outcome in 109 women presenting before 35 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment, excluding those with known abnormal scan findings on day of enrolment Table S12 Characteristics of 123 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia (PE), according to subsequent birth weight of infant Table S13 Delivery characteristics and maternal and neonatal outcome of 123 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia (PE), according to subsequent birth weight of infant Table S14 Individual biomarker areas under receiver–operating characteristics curves (AUCs) when sampled between 35 + 0 and 36 + 6 weeks' gestation

Table S13 Delivery characteristics and maternal and neonatal outcome of 123 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia (PE), according to subsequent birth weight of infant Table S14 Individual biomarker areas under receiver–operating characteristics curves (AUCs) when sampled between 35 + 0 and 36 + 6 weeks' gestation Table S15 Predictive performance of individual biomarkers and combinations (derived from logistic regression) for prediction of small‐for‐gestational age (SGA) < 3rd centile and < 10th centile in 123 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia Table S16 Predictive performance of individual indicators and their combinations, to predict small‐for‐gestational age (SGA) < 3rd customized birth‐weight centile in 53 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment Table S17 Predictive performance of individual indicators and their combinations, to predict small‐for‐gestational age (SGA) < 10th customized birth‐weight centile in 53 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment Table S18 Predictive performance of individual indicators and their combinations, to predict adverse perinatal outcome in 53 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment

Table S17 Predictive performance of individual indicators and their combinations, to predict small‐for‐gestational age (SGA) < 10th customized birth‐weight centile in 53 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment Table S18 Predictive performance of individual indicators and their combinations, to predict adverse perinatal outcome in 53 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment Click here for additional data file. ACKNOWLEDGMENT We thank the doctors, midwives and sonographers at the study centers for their work, and the women who participated.

Table S18 Predictive performance of individual indicators and their combinations, to predict adverse perinatal outcome in 53 women presenting between 35 + 0 and 36 + 6 weeks' gestation with suspected pre‐eclampsia, who underwent ultrasound examination within 14 days of enrolment Click here for additional data file. ACKNOWLEDGMENT We thank the doctors, midwives and sonographers at the study centers for their work, and the women who participated. Disclosures We acknowledge funding support from Tommy's Charity (registered charity no 1060508 and SCO39280), the National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy's and St Thomas' NHS Foundation Trust and King's College London, UK and Alere (San Diego, California). This was an investigator‐led study and no funder had any role in study design, patient recruitment, data collection, analysis or interpretation, nor in the writing of the manuscript or the decision to submit it for publication. No author has been paid to write this article by these or any other funders. As corresponding author, L.C.C. had full access to all the data in the study. J.M. is supported by Action Medical Research Endowment Fund, the Manchester Biomedical Research Centre and the Greater Manchester Comprehensive Local Research Network. P.T.S's salary is supported by Tommy's Charity. L.C.K. is supported by a Science Foundation Ireland Program Grant for INFANT (12/RC/2272). C.W.G.R., A.H.S., N.S. and J.M. were paid as consultants for Alere prior to 2013; A.H.S. has also been paid as a consultant for Roche and PerkinElmer. R.N. was principal investigator on a study investigating biomarker prediction of PE funded by an unrestricted research grant from Alere Inc., San Diego, CA, USA (www.alere.com), to the universities in the SCOPE consortium.

d quality‐improvement measures Findings from the first cycle of the audit relating to performance of the department as a whole were fed back to the sonography team. An individualized written report was also generated for each of the sonographers, who were given the opportunity to discuss any concerns or training needs. Following this, a number of actions and interventions were agreed and undertaken, including the production and dissemination of updated and formalized anomaly scan departmental protocol, the production for each scan room of aide memoire posters of the FASP base image menu (six views) required for storage, the introduction of guidance on image storage as well as a cardiac scanning protocol with specific hands‐on training sessions, and providing supported practice for individuals with specific training needs identified in Cycle 1 of the audit. To assess the effect of quality‐improvement interventions on the performance of sonographers in the department, all anomaly scans acquired in 2014 were audited (follow‐up audit) following the same procedures used to analyze the 2013 data.

3 (−2.63 to − 0.62) 4.80 − 11.0 to 7.77 − 0.12 (−0.20 to − 0.03) 0.42 AC 90 − 0.59 (−1.77 to 0.60) 5.65 − 11.7 to 10.45 − 0.03 (−0.13 to 0.07) 0.50 FL 90 − 0.35 (−0.64 to − 0.07) 1.37 − 3.03 to 2.33 − 0.11 (−0.21 to 0) 0.50 AC, abdominal circumference; BPD, biparietal diameter; FL, femur length; HC, head circumference. Femur length The only parameter for which the SDs and the differences were constant throughout the range of measurements was FL (Figures 2 and 3). The mean difference between the ultrasound measurements for each fetus obtained by the two different examiners is presented in Table 2. On average, the measurement of femur length by Examiner 1 differed from the measurement made by Examiner 2 by –0.13 mm (95% CI, –0.28 to 0.03 mm) (Table 2). The 95% limits of agreement ranged from − 3.03 to 2.78 mm. This meant that the measurements of Examiner 2 were likely to be within − 3.03 to 2.78 mm of Examiner 1's measurements 95% of the time, a difference that corresponds to a ± 1–1.5-week variation in gestational age estimation.

95% CI, –0.28 to 0.03 mm) (Table 2). The 95% limits of agreement ranged from − 3.03 to 2.78 mm. This meant that the measurements of Examiner 2 were likely to be within − 3.03 to 2.78 mm of Examiner 1's measurements 95% of the time, a difference that corresponds to a ± 1–1.5-week variation in gestational age estimation. Biparietal diameter, head circumference and abdominal circumference To account for the increase in variation that occurred with the increase in magnitude of the measurements, the values for BPD, HC and AC were log-transformed to calculate the limits of agreement. These were then back-transformed so they could be related to the original scale of measurement18, 20. For HC and BPD, 95% of measurements by Examiner 2 could be expected to be 0.95–1.05 times the measurement by Examiner 1. This meant that the measurements by Examiner 2 could differ by 5% above or below that of Examiner 1. If the HC measurement by Examiner 1 was 116 mm (minimum HC), corresponding to a gestational age of 15 + 3 weeks, we would expect the measurement by Examiner 2 to be within ± 5.8 mm 95% of the time. This corresponds to an acceptable variation in estimated gestational age of ± 3 days (15 + 0 and 15 + 6 weeks). The variation increased with the size of the measurement. Therefore, if the HC measurement obtained by Examiner 1 was 284 mm, we would expect the measurement by Examiner 2 to be within ± 14.2 mm 95% of the time. An HC of 284 mm corresponds to a gestational age of 30 + 0 weeks, and ± 14.2 mm to a possible difference of 1.5 weeks, which was considered as just clinically acceptable. Similarly, for a measurement of 344 mm (the maximum HC, corresponding to 39 + 3 weeks) we would expect a possible difference of 4 weeks, larger than clinically acceptable.

INTRODUCTION A mid‐trimester fetal anatomy ultrasound scan (or ‘anomaly scan’) is offered to pregnant women in most developed countries. Clinical guidelines, such as those defined by the Fetal Anomaly Screening Programme (FASP)1 in the UK or the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG)2, specify protocols for such screening. For instance, FASP recommends ultrasound image capture of six views: the head in the transventricular (TV) and the transcerebellar (TC) planes, femur length (FL), abdominal circumference (AC), the lips in the coronal view and the spine. Additionally, FASP recommends visualization (though it does not mandate image capture) of other fetal structures, such as the heart, kidneys and limbs. Such protocols aim to guide practice of fetal ultrasonography to provide a ‘checklist’ of what imaging planes to capture. In principle, if all standard imaging planes are examined, detection of anomalies should be maximized. The quality of archived images reflects overall quality of fetal anatomical survey. Whilst this correlation is not absolute, the concern is that failure to archive images or poor image quality could support medicolegal claims.

ple, if all standard imaging planes are examined, detection of anomalies should be maximized. The quality of archived images reflects overall quality of fetal anatomical survey. Whilst this correlation is not absolute, the concern is that failure to archive images or poor image quality could support medicolegal claims. Regular audit, peer review and quality assurance procedures are recognized to improve and sustain good practice3, 4, 5. However, routinely auditing images is a resource‐intensive administrative task; it is time‐consuming, tedious and takes highly trained staff away from clinical work. Consequently, a comprehensive ultrasound image audit is often undertaken only in a ‘reactive’ manner to a potential ‘screening failure’, such as in response to an unanticipated major anomaly at birth. Given the relative rarity of major anomalies, such a reactive approach is unlikely to highlight poor performance of a single operator or a department. Even when departments do perform a regular image audit, this is usually performed on a limited number of scans per sonographer. Such a selective audit is incomplete and may not provide adequate insight into sonographer performance6. Finally, however desirable a comprehensive image audit is, its resource‐intensive nature means that findings and feedback may seem no longer relevant to current practice5, 7.

limited number of scans per sonographer. Such a selective audit is incomplete and may not provide adequate insight into sonographer performance6. Finally, however desirable a comprehensive image audit is, its resource‐intensive nature means that findings and feedback may seem no longer relevant to current practice5, 7. In theory, an automated software‐based ultrasound image audit could be a solution for full (100%) clinical auditing. Putting aside technical challenges to achieve this, it is important to first ascertain whether a comprehensive image‐quality audit can improve performance. The aim of this study was to determine whether a full clinical audit improves performance in terms of image acquisition/storage and image quality in a large maternity ultrasound department. METHODS Overview of audit cycle This quality‐improvement study comprised a retrospective baseline audit (Cycle 1) of anonymized routine fetal anomaly scans performed over a 12‐month period in 2013 at a large teaching hospital with over 6500 births per annum. This was followed by feedback to sonographers of individual and departmental performance scores and implementation of a number of targeted interventions to improve quality of imaging. In the subsequent 12‐month period, a re‐audit (Cycle 2) using the same quality criteria was performed in order to evaluate the impact of the changes.

owed by feedback to sonographers of individual and departmental performance scores and implementation of a number of targeted interventions to improve quality of imaging. In the subsequent 12‐month period, a re‐audit (Cycle 2) using the same quality criteria was performed in order to evaluate the impact of the changes. Audit process Ultrasound images and meta‐data (scan date, ultrasound machine used, sonographer ID) were extracted from the hospital database (ViewPoint, GE Medical Systems, Zipf, Austria) and anonymized to remove all patient identifiable information. Images were audited by 12 experienced sonographers, currently practicing in different hospitals in the UK, using a standard audit pro forma including a custom‐built easy‐to‐use online interface created by Intelligent Ultrasound Ltd (Milton Park, Oxfordshire, UK). This facilitated the auditing of scan completeness (assessment of whether a scan had the complete minimum set of required images) and image quality (assessment of recorded images against a scoring system). Before commencing the study, all assessors participated in a training day to ensure familiarity with imaging protocols, quality criteria and software, and had the opportunity to practice quality assessment using a large number of case examples. Scan completeness and image quality were measured at the level of the department and the individual sonographer for all departmental sonographers who undertook at least 30 anomaly scans in each cycle.

Before commencing the study, all assessors participated in a training day to ensure familiarity with imaging protocols, quality criteria and software, and had the opportunity to practice quality assessment using a large number of case examples. Scan completeness and image quality were measured at the level of the department and the individual sonographer for all departmental sonographers who undertook at least 30 anomaly scans in each cycle. Scan completeness According to the UK FASP guidelines1, a complete fetal anatomy examination should include images of the following six views: head in the TV plane, head in the TC plane, AC, FL, spine and face (lips) in the coronal plane. For the purposes of this audit, if at least one required image was missing from an examination, it was considered incomplete. Scan completeness was defined as the proportion of the total number of audited patient scans that were complete (i.e. all required images were obtained). View completeness was defined as the proportion of scans that had at least one image for a particular view.

missing from an examination, it was considered incomplete. Scan completeness was defined as the proportion of the total number of audited patient scans that were complete (i.e. all required images were obtained). View completeness was defined as the proportion of scans that had at least one image for a particular view. Image‐quality scoring The quality of images of the head TV, head TC, AC, FL, face (lips) and spine views was also audited. The quality of each image was assessed against a number of criteria (Table 1) based on studies and guidelines, and reflective of the protocol1, 8, 9. A score for each image was computed as the sum of satisfied criteria in a view; the maximum achievable score was 7 for the head TV, head TC and spine images, 6 for the AC images and 4 for the FL and coronal face images. As the number of criteria varied according to view, each score was divided by the maximum score, to give a normalized score between 0 and 1. This allowed computation of an average image‐quality score overall and for each view. The median image‐quality score for each sonographer was also computed. Table 1 Criteria for quality assessment of ultrasound images of head transventricular (TV), head transcerebellar (TC), abdomen, femur, spine and coronal face views

Image‐quality scoring The quality of images of the head TV, head TC, AC, FL, face (lips) and spine views was also audited. The quality of each image was assessed against a number of criteria (Table 1) based on studies and guidelines, and reflective of the protocol1, 8, 9. A score for each image was computed as the sum of satisfied criteria in a view; the maximum achievable score was 7 for the head TV, head TC and spine images, 6 for the AC images and 4 for the FL and coronal face images. As the number of criteria varied according to view, each score was divided by the maximum score, to give a normalized score between 0 and 1. This allowed computation of an average image‐quality score overall and for each view. The median image‐quality score for each sonographer was also computed. Table 1 Criteria for quality assessment of ultrasound images of head transventricular (TV), head transcerebellar (TC), abdomen, femur, spine and coronal face views Head TV Head TC Abdomen Femur Spine Coronal face Symmetrical plane with midline echo of falx dividing skull Symmetrical plane with midline echo of falx dividing skull Stomach bubble visible Both ends of ossified diaphysis clear Magnification at least 30% of screen Upper lip visible Cavum septum visible one‐third of way along midline falx Cavum septum visible one‐third of way along midline falx Umbilical vein one‐third of way along anteroposterior diameter, at level of portal sinus Angle of insonation 45–90° Continuity intact and posterior skin edge visible Two nostrils visible Cerebellum not visible Cerebellum visible Circular plane Magnification at least 30% of screen Alignment of vertebrae visible Two lip angles visible Posterior ventricle visible at level of atrium Cisterna magna visible Kidneys not visible Calipers placed on clear ends of diaphysis Amniotic fluid visible beyond skin Adequate magnification Magnification at least 30% of screen Magnification at least 30% of screen Magnification at least 30% of screen Lower (sacrum) visible Head circumference calipers placed appropriately on outer parts of skull Calipers placed correctly on outer limits of cerebellar hemispheres Abdominal circumference calipers placed appropriately on outer parts of abdomen Middle (thoracic/lumbar) visible Calipers measuring lateral ventricles at level of atrium Calipers placed correctly on outer limits of cisterna magna Upper (cervical/thoracic) visible Cycle 1: feedback and quality‐improvement measures Findings from the first cycle of the audit relating to performance of the department as a whole were fed back to the sonography team. An individualized written report was also generated for each of the sonographers, who were given the opportunity to discuss any concerns or training needs.

Following this, a number of actions and interventions were agreed and undertaken, including the production and dissemination of updated and formalized anomaly scan departmental protocol, the production for each scan room of aide memoire posters of the FASP base image menu (six views) required for storage, the introduction of guidance on image storage as well as a cardiac scanning protocol with specific hands‐on training sessions, and providing supported practice for individuals with specific training needs identified in Cycle 1 of the audit. To assess the effect of quality‐improvement interventions on the performance of sonographers in the department, all anomaly scans acquired in 2014 were audited (follow‐up audit) following the same procedures used to analyze the 2013 data. Measuring interobserver agreement In order to measure agreement, 10% of the audited baseline and follow‐up images were selected by computer randomization and assessed by two independent reviewers. Agreement in scan completeness was measured as the percentage of agreed images for each view (i.e. if Reviewer 1 and Reviewer 2 agreed on the view type) and a kappa statistic was calculated. The ultrasound views for which reviewers had the highest disagreement were investigated. Variability in image‐quality assessment was evaluated by comparing overall image scores. The variability for each criterion in all views was assessed to investigate which criteria contributed most to the overall variability of reviewers.

trasound views for which reviewers had the highest disagreement were investigated. Variability in image‐quality assessment was evaluated by comparing overall image scores. The variability for each criterion in all views was assessed to investigate which criteria contributed most to the overall variability of reviewers. Statistical analysis The chi‐squared test was used to assess whether changes in scan completeness between the two cycles were statistically significant. For image‐quality scoring, paired comparisons of the normalized score (from 0 to 1) were performed using the Wilcoxon matched‐pairs signed‐ranks test, as normality tests suggested evidence of a non‐normal distribution. Two‐sided t‐tests are reported and the significance level was set at < 0.05. Analysis was performed using Stata 15 (StataCorp LLC, College Station, TX, USA).

malized score (from 0 to 1) were performed using the Wilcoxon matched‐pairs signed‐ranks test, as normality tests suggested evidence of a non‐normal distribution. Two‐sided t‐tests are reported and the significance level was set at < 0.05. Analysis was performed using Stata 15 (StataCorp LLC, College Station, TX, USA). RESULTS In Cycle 1, 103 501 images from 6257 anomaly examinations performed by 22 sonographers were assessed. Sonographers had varying working patterns (full time, part time or locums). The median number of scans per sonographer was 237 and the median number of audited images per scan was 16. In Cycle 2, 153 557 images were evaluated from 6406 anomaly examinations performed by 25 sonographers, of whom 20 also performed examinations in Cycle 1. The median number of scans per sonographer in Cycle 2 was 189 and the median number of audited images per scan was 24. All scans were performed between 18 + 0 and 22 + 6 weeks of gestation (Figure 1). Scans were acquired using a GE Voluson E6, GE Voluson E8, GE Voluson 730 (GE Medical Systems) and Hitachi Aloka ProSound Alpha 10 (Twinsburg, OH, USA) machines, none of which had automatic caliper placement. Figure 1 Frequency of fetal anatomical ultrasound examinations performed in a large maternity ultrasound department in 2013 () and 2014 (), according to gestational age.

RESULTS In Cycle 1, 103 501 images from 6257 anomaly examinations performed by 22 sonographers were assessed. Sonographers had varying working patterns (full time, part time or locums). The median number of scans per sonographer was 237 and the median number of audited images per scan was 16. In Cycle 2, 153 557 images were evaluated from 6406 anomaly examinations performed by 25 sonographers, of whom 20 also performed examinations in Cycle 1. The median number of scans per sonographer in Cycle 2 was 189 and the median number of audited images per scan was 24. All scans were performed between 18 + 0 and 22 + 6 weeks of gestation (Figure 1). Scans were acquired using a GE Voluson E6, GE Voluson E8, GE Voluson 730 (GE Medical Systems) and Hitachi Aloka ProSound Alpha 10 (Twinsburg, OH, USA) machines, none of which had automatic caliper placement. Figure 1 Frequency of fetal anatomical ultrasound examinations performed in a large maternity ultrasound department in 2013 () and 2014 (), according to gestational age. UOG-20144-FIG-0001-bScan‐completeness audit Overall scan completeness in the baseline audit was 72%, which improved to 78% in the follow‐up audit (χ2 = 25.99, P < 0.001) with improved completeness for all views. Table 2 summarizes the overall scan completeness and completeness per view in the baseline and follow‐up audits. Figure 2 shows scan completeness for all included sonographers (n = 20) who acquired at least 30 scans in each year. For example, sonographer 1 undertook 138 scans in Cycle 1, of which 113 were complete. The scan‐completeness audit demonstrated that the improvement was particularly noticeable for those sonographers who had the lowest completeness in the baseline audit. Sonographers whose scan completeness in Cycle 1 was less than 50% had a mean scan completeness improvement from 19.0% to 60.3% in Cycle 2. While this could be purely due to ‘regression to the mean’, this is unlikely given the consistency of improvements seen.

phers who had the lowest completeness in the baseline audit. Sonographers whose scan completeness in Cycle 1 was less than 50% had a mean scan completeness improvement from 19.0% to 60.3% in Cycle 2. While this could be purely due to ‘regression to the mean’, this is unlikely given the consistency of improvements seen. Table 2 Overall and per‐view scan completeness and quality score in baseline and follow‐up audits of fetal anatomical ultrasound examinations View Baseline audit Follow‐up audit Difference (%)* Completeness (%) Head TV 89.0 92.8 +3.9 Head TC 87.4 90.7 +3.4 Abdomen 90.9 93.3 +2.4 Femur 89.9 92.9 +3.0 Spine 84.7 91.0 +6.4 Coronal face 83.5 88.9 +5.5 Overall completeness† 72.4 77.8 +5.3 Quality score Head TV 0.81 0.88 +0.07 Head TC 0.84 0.90 +0.07 Abdomen 0.86 0.87 +0.01 Femur 0.85 0.89 +0.04 Spine 0.68 0.77 +0.09 Coronal face 0.60 0.68 +0.08 Median overall quality score 0.83 0.86 +0.03 * Difference = (Follow‐up – Baseline). † Proportion of complete scans for all views. TC, transcerebellar; TV, transventricular. Figure 2 Scan completeness for fetal anatomical ultrasound examinations performed by 20 sonographers, in baseline () and follow‐up () audits.

Completeness (%) Head TV 89.0 92.8 +3.9 Head TC 87.4 90.7 +3.4 Abdomen 90.9 93.3 +2.4 Femur 89.9 92.9 +3.0 Spine 84.7 91.0 +6.4 Coronal face 83.5 88.9 +5.5 Overall completeness† 72.4 77.8 +5.3 Quality score Head TV 0.81 0.88 +0.07 Head TC 0.84 0.90 +0.07 Abdomen 0.86 0.87 +0.01 Femur 0.85 0.89 +0.04 Spine 0.68 0.77 +0.09 Coronal face 0.60 0.68 +0.08 Median overall quality score 0.83 0.86 +0.03 * Difference = (Follow‐up – Baseline). † Proportion of complete scans for all views. TC, transcerebellar; TV, transventricular. Figure 2 Scan completeness for fetal anatomical ultrasound examinations performed by 20 sonographers, in baseline () and follow‐up () audits. UOG-20144-FIG-0002-bImage‐quality audit One sonographer was excluded from the image‐quality analysis due to having insufficient images for assessment in Cycle 1. For the remaining 19 sonographers who acquired at least 30 scans in each year, the median image‐quality score (for all images in all views) improved from 0.83 (interquartile range (IQR), 0.69–0.89) in 2013 to 0.86 (IQR, 0.76–0.91) in 2014 (P < 0.001; Table 2). The median normalized image‐quality scores for these 19 sonographers demonstrate that the greatest improvement was obtained for those with the lowest baseline audit score (Figure 3). For the nine sonographers with normalized image‐quality score < 0.80 in Cycle 1, the average quality score improved from 0.70 to 0.80 in Cycle 2, while those who had good image‐quality score at baseline performed similarly in the follow‐up audit.

t was obtained for those with the lowest baseline audit score (Figure 3). For the nine sonographers with normalized image‐quality score < 0.80 in Cycle 1, the average quality score improved from 0.70 to 0.80 in Cycle 2, while those who had good image‐quality score at baseline performed similarly in the follow‐up audit. Figure 3 Normalized image‐quality score for fetal anatomical ultrasound examinations performed by 19 sonographers, in baseline () and follow‐up () audits. UOG-20144-FIG-0003-bInterobserver agreement The overall interobserver agreement between two reviewers for scan completeness was 95.1%. The agreement varied slightly between the different views, as shown by the head TV and spine images having the least agreement between sonographers (92.1% and 93.7%, respectively; Table 3). The overall mean interobserver agreement for assessing image quality was 82.7%. Table 4 shows the interobserver agreement for quality score for each view and each criterion. Table 3 Interobserver agreement of 20 sonographers for assessment of fetal anatomical ultrasound scan completeness View Agreement (%) Head TV 92.1 Head TC 96.5 Abdomen 96.5 Femur 98.8 Spine 93.7 Coronal face 96.8 Overall 95.1 TC, transcerebellar; TV, transventricular. Table 4 Interobserver agreement in assessment of quality criteria of ultrasound images of six standard views

Table 3 Interobserver agreement of 20 sonographers for assessment of fetal anatomical ultrasound scan completeness View Agreement (%) Head TV 92.1 Head TC 96.5 Abdomen 96.5 Femur 98.8 Spine 93.7 Coronal face 96.8 Overall 95.1 TC, transcerebellar; TV, transventricular. Table 4 Interobserver agreement in assessment of quality criteria of ultrasound images of six standard views View/criterion Agreement (%) Head TV 87.0 Ventricle calipers 75.8 Head circumference calipers 82.0 Symmetrical circular brain 84.2 Cavum septum visible 86.1 Ventricles visible 88.8 Magnification 92.5 No cerebellum 99.3 Head TC 88.0 Cavum septum visible 79.5 Calipers cisterna magna 81.4 Symmetrical circular brain 88.4 Magnification 88.9 Calipers cerebellar diameter 89.2 Cerebellum visible 93.4 Cisterna magna visible 95.0 Abdomen 83.6 Abdominal circumference calipers 69.4 Umbilical vein visible 77.5 Circular 83.4 Magnification 86.5 No kidney 90.1 Stomach visible 94.4 Femur 82.2 Magnification 68.0 Femur length calipers 78.2 Clear femoral ends 88.1 Angle of insonation 94.3 Spine 75.4 Amniotic fluid visible beyond the skin 69.1 Magnification 73.2 Continuity intact and posterior skin edge visible 73.2 Alignment of vertebrae visible 74.5 Upper (cervical/ thoracic) visible 78.1 Lower (sacrum) visible 78.4 Middle (thoracic/lumbar) visible 81.3 Coronal face 80.0 Magnification 72.1 Upper lip visible 79.5 Two lip angles visible 81.2 Two nostrils visible 87.0 Overall agreement 82.7 Criteria for each view are sorted in ascending order of agreement. TC, transcerebellar; TV, transventricular.

View/criterion Agreement (%) Head TV 87.0 Ventricle calipers 75.8 Head circumference calipers 82.0 Symmetrical circular brain 84.2 Cavum septum visible 86.1 Ventricles visible 88.8 Magnification 92.5 No cerebellum 99.3 Head TC 88.0 Cavum septum visible 79.5 Calipers cisterna magna 81.4 Symmetrical circular brain 88.4 Magnification 88.9 Calipers cerebellar diameter 89.2 Cerebellum visible 93.4 Cisterna magna visible 95.0 Abdomen 83.6 Abdominal circumference calipers 69.4 Umbilical vein visible 77.5 Circular 83.4 Magnification 86.5 No kidney 90.1 Stomach visible 94.4 Femur 82.2 Magnification 68.0 Femur length calipers 78.2 Clear femoral ends 88.1 Angle of insonation 94.3 Spine 75.4 Amniotic fluid visible beyond the skin 69.1 Magnification 73.2 Continuity intact and posterior skin edge visible 73.2 Alignment of vertebrae visible 74.5 Upper (cervical/ thoracic) visible 78.1 Lower (sacrum) visible 78.4 Middle (thoracic/lumbar) visible 81.3 Coronal face 80.0 Magnification 72.1 Upper lip visible 79.5 Two lip angles visible 81.2 Two nostrils visible 87.0 Overall agreement 82.7 Criteria for each view are sorted in ascending order of agreement. TC, transcerebellar; TV, transventricular. Audit time The mean reviewing time per image was 24 ± 20 s, but this varied between views, as seen in the high SD. The mean time for review per scan was 8.4 ± 6.1 min for assessment of both completeness (mean, 2.0 ± 1.2 min) and grading of gradable images (6.4 ± 5.5 min). Overall, the assessment of Cycle 1 took approximately 876 h while Cycle 2 took about 897 h.

was 24 ± 20 s, but this varied between views, as seen in the high SD. The mean time for review per scan was 8.4 ± 6.1 min for assessment of both completeness (mean, 2.0 ± 1.2 min) and grading of gradable images (6.4 ± 5.5 min). Overall, the assessment of Cycle 1 took approximately 876 h while Cycle 2 took about 897 h. DISCUSSION In this study, we assessed the effect of a full audit cycle involving large‐scale fetal anomaly scan audit on the performance of sonographers. In this audit, fetal scans were assessed manually by experts to evaluate scan completeness and image quality. A full‐year baseline audit was performed to establish the sonographers' baseline performance. Actions were implemented in the department in the following year and a second follow‐up audit was performed. We found that both scan completeness and image quality improved significantly between the audit cycles.

mage quality. A full‐year baseline audit was performed to establish the sonographers' baseline performance. Actions were implemented in the department in the following year and a second follow‐up audit was performed. We found that both scan completeness and image quality improved significantly between the audit cycles. Although the improvement was by a relatively small margin as a department, Cycle 2 demonstrated large improvements for those sonographers who performed least well in the first audit cycle. Sonographers whose scan completeness in the baseline audit was less than 50% had a mean scan completeness improvement from 19.0% to 62.3% in the follow‐up audit, including two sonographers (sonographers 19 and 20) who had 0% scan completeness in the baseline audit. Similarly, normalized image‐quality score for sonographers whose baseline image‐quality score was less than 0.80 improved from a mean of 0.70 to 0.80 in the follow‐up audit. In contrast, sonographers who had good scan completeness and image quality at baseline performed similarly in the follow‐up audit.

baseline audit. Similarly, normalized image‐quality score for sonographers whose baseline image‐quality score was less than 0.80 improved from a mean of 0.70 to 0.80 in the follow‐up audit. In contrast, sonographers who had good scan completeness and image quality at baseline performed similarly in the follow‐up audit. Reduction in variation in practice and sustaining good practice are both recognized factors in upholding patient safety10, 11; this has also been demonstrated in fetal ultrasound. Although not evaluated as part of this study, the observed individual improvements could enhance overall screening performance of an imaging department. It is important to note that the finding of improvement in scan completeness was observed not only overall but in each individual anatomical view, and that the overall scan completeness is lower than view completeness for each view, due to the rigorous way in which we defined scan completeness.

f an imaging department. It is important to note that the finding of improvement in scan completeness was observed not only overall but in each individual anatomical view, and that the overall scan completeness is lower than view completeness for each view, due to the rigorous way in which we defined scan completeness. Interobserver agreement was high for the assessment of scan completeness; it was lowest for the head TV view, which may be because the original local protocol was unclear as to whether the head TV or transthalamic (TT) view should be stored, and therefore some recorded head views were in the TT view. Although TV and TT planes are very close to each other12, this could introduce some disagreement between reviewers. The interobserver agreement was higher for completeness compared with assessment of image quality. This is likely to be due to the more complex nature of image‐quality assessment, which includes assessing several criteria per view. It is also evident that assessment of caliper placement is not as reproducible as other criteria (Table 4). For instance, the agreement on the placement of AC calipers was less than 70% and the worst among other abdominal criteria.

nature of image‐quality assessment, which includes assessing several criteria per view. It is also evident that assessment of caliper placement is not as reproducible as other criteria (Table 4). For instance, the agreement on the placement of AC calipers was less than 70% and the worst among other abdominal criteria. One of the key strengths of this study is that the entire scan output of a single department was assessed, removing any potential biases in selection of examinations for review and offering the most comprehensive assessment of both departmental and individual performance. A limitation of this approach is that it is very labor intensive. Although it is possible that a retrospective audit of a small number (e.g. 10–20 cases) of consecutive scans would suffice, this would provide only limited understanding of individual and departmental performance. Another option, self‐scoring, in which sonographers appraise their own images, has been shown previously to be feasible13. However, this self‐scoring was undertaken within a framework of additional, independent scoring; it is not known whether this would be of benefit in the absence of such external review13.

formance. Another option, self‐scoring, in which sonographers appraise their own images, has been shown previously to be feasible13. However, this self‐scoring was undertaken within a framework of additional, independent scoring; it is not known whether this would be of benefit in the absence of such external review13. In our study, independent visual assessment was performed by experienced sonographers. This was based on an objective, criteria‐based scoring system that was shown to have a high degree of reproducibility for image scoring. High reproducibility (kappa, 92.7) for image scoring methods shown in a previous study8 was also seen in the current study, and confirms the high level of training of the reviewers conducting the audit. Despite this, the assessment of placement of calipers in the four biometry views typically had a low agreement between reviewers (Table 4); the significant contribution of caliper placement to variability is in keeping with the findings of previous studies, such as that of Sarris et al.14. In addition, although the femur is a long straight object, there was significant disagreement between reviewers when assessing magnification of the femur images. In contrast, criteria that assessed if an object is not visible on an image, for example, no cerebellum in the head TV view, had high agreement between reviewers.

ddition, although the femur is a long straight object, there was significant disagreement between reviewers when assessing magnification of the femur images. In contrast, criteria that assessed if an object is not visible on an image, for example, no cerebellum in the head TV view, had high agreement between reviewers. One potential limitation of this study was that the two audit cycles were back‐to‐back. This was to allow seamless observation of practice over time. It is highly likely that, had we allowed a period of time to pass before re‐audit to ensure that changes in practice in response to Cycle 1 were more embedded, greater improvements would have been seen. Given the described audit process, it is possible that some of the observed improvements in performance may have related more to sonographer awareness of ongoing audit and knowledge of the specific criteria used to evaluate performance. However, a longer study period could have been subject to confounding due to other unmeasured changes. Another limitation is that we assessed only six views, based on the recommendations for practice in the UK; we, of course, recognize that other scanning protocols suggest a larger number of images to be stored. However, the improvements seen in this smaller number of views should also apply. Although imaging of cardiac views was introduced in 2014, this was on the basis of best practice and they were not included in the analysis as they were not assessed in Cycle 1.

ng protocols suggest a larger number of images to be stored. However, the improvements seen in this smaller number of views should also apply. Although imaging of cardiac views was introduced in 2014, this was on the basis of best practice and they were not included in the analysis as they were not assessed in Cycle 1. In conclusion, we have shown that large‐scale clinical audit, coupled with implementation of targeted changes and feedback to sonographers, can lead to improvements in image quality on the mid‐trimester anomaly scan, in terms of both completeness of scans and image quality. However, while quality improvement is possible, such comprehensive manual audit in a high‐throughput clinical setting is a very labor‐intensive process and this would be a major barrier for implementation in routine practice. Ongoing work on automated image analysis15 and further research into automated image recognition would open up the possibility of more rapid audit processes, and the current work provides evidence that this could be effective.

r‐intensive process and this would be a major barrier for implementation in routine practice. Ongoing work on automated image analysis15 and further research into automated image recognition would open up the possibility of more rapid audit processes, and the current work provides evidence that this could be effective. ACKNOWLEDGMENTS This research was conducted as part of the AQABUS project, a joint partnership between the University of Oxford and Intelligent Ultrasound Ltd funded by Innovate UK (Project 101684) and the UK Engineering and Physical Sciences Research Council (EP/L505316/1). The manual audit tool was developed by Intelligent Ultrasound Ltd. J.A.N. and A.T.P. are cofounders of, and consult for, Intelligent Ultrasound Ltd, a company that develops software and systems to improve ultrasound imaging and services. A.T.P. is supported by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC). The support of the 12 reviewers who performed the audit is gratefully acknowledged.

CONTRIBUTION What are the novel findings of this work? A detailed, novel analysis of full‐length routine fetal growth scans and sonographer eye tracking has shown that operators are at risk of expected‐value bias when acquiring fetal biometry measurements. What are the clinical implications of this work? When making clinical decisions, clinicians should be aware that estimated fetal weight may be inaccurate due to expected‐value bias. Ultrasound operators should be aware of this potential bias while performing biometric measurements.

A detailed, novel analysis of full‐length routine fetal growth scans and sonographer eye tracking has shown that operators are at risk of expected‐value bias when acquiring fetal biometry measurements. What are the clinical implications of this work? When making clinical decisions, clinicians should be aware that estimated fetal weight may be inaccurate due to expected‐value bias. Ultrasound operators should be aware of this potential bias while performing biometric measurements. INTRODUCTION In science, the accuracy of measurement is a crucial prerequisite for correct interpretation of results. There are many reasons for inaccurate measurement and one that is relatively easy to overcome is the observer bias, which is the tendency to see what we expect to see1. The observer bias is also known as expected‐value bias, detection bias, observer–expectancy effect, expectancy bias, observer effect or ascertainment bias. This bias may occur if the observer has a preconceived idea of what a measurement ought to be, leading to adjustments of the readings. Hróbjartsson and colleagues2 undertook a systematic review quantifying the impact of observer bias, by comparing estimates between studies in which outcome assessors were blinded to the intervention and those in which outcome assessors were not blinded. For clinical trials that used measurement scale outcomes, non‐blinded outcome assessment exaggerated the effect size by as much as 68%2. In randomized trials, blinding is used to reduce bias and usually involves preventing knowledge of which intervention or control is being received by a study participant3, 4. Day and Altman5 highlight that blinding is important in other types of research too, such as evaluation of the performance of a diagnostic test and reproducibility of measurement techniques. Blinding makes it difficult to bias results intentionally or unintentionally and so helps to ensure the credibility of measurements5. Recently, a review of systematic error and cognitive bias in obstetric ultrasound suggested that expectation bias is pertinent to obstetric ultrasound studies6.

f measurement techniques. Blinding makes it difficult to bias results intentionally or unintentionally and so helps to ensure the credibility of measurements5. Recently, a review of systematic error and cognitive bias in obstetric ultrasound suggested that expectation bias is pertinent to obstetric ultrasound studies6. In contrast to trials, measurement blinding is not usually carried out in day‐to‐day clinical management. This may be of particular relevance in fetal growth assessment, which looks for aberrations from normally expected growth patterns; however, blinding of the examiner to the gestational age of the pregnancy to avoid the effect of clinician bias is rarely practiced. During clinical assessment of fundal height, the guidance suggests that caregivers should hold the tape in a way that the measurement cannot be seen7. This is however not usually the case in ultrasound assessment; during a routine growth scan, comprising the three standard biometric plane measurements of head circumference (HC), abdominal circumference (AC) and femur length (FL)8, the ultrasound machine will usually display the reading value (circumference or length, in mm or cm) as well as an observed gestational age (in weeks + days) corresponding to the measurement (Figure 1). This can lead to an observer (or expected‐value) bias, which means that the operator may adjust the circumference or length so that the observed gestational age matches the gestational age calculated previously by dating. In turn, this may lead to a biased fetal growth estimation. The use of blinding in this scenario would overcome such bias. Although blinding of the operator to the actual gestational age or to the machine‐displayed values during growth scan assessment has been done in some studies9, 10, measurement blinding is rarely used in routine clinical practice11.

fetal growth estimation. The use of blinding in this scenario would overcome such bias. Although blinding of the operator to the actual gestational age or to the machine‐displayed values during growth scan assessment has been done in some studies9, 10, measurement blinding is rarely used in routine clinical practice11. Figure 1 Occurrence of expected‐value bias during measurement of fetal abdominal circumference at 28 + 0 weeks' gestation. Red rectangle outlines measurement box and green dot has been added to represent operator eye focus (not visible to operator during scan). (a) Caliper adjustment in progress. (b) Operator eye fixation on measurement box detected, suggesting biased measurement. (c) Measurement accepted. UOG-21929-FIG-0001-cIn this study we aimed to evaluate the incidence of expected‐value bias in routine fetal growth scans and assess its impact on standard biometric measurements.

Figure 1 Occurrence of expected‐value bias during measurement of fetal abdominal circumference at 28 + 0 weeks' gestation. Red rectangle outlines measurement box and green dot has been added to represent operator eye focus (not visible to operator during scan). (a) Caliper adjustment in progress. (b) Operator eye fixation on measurement box detected, suggesting biased measurement. (c) Measurement accepted. UOG-21929-FIG-0001-cIn this study we aimed to evaluate the incidence of expected‐value bias in routine fetal growth scans and assess its impact on standard biometric measurements. METHODS This was a prospective study of routine ultrasound scans performed between May 2018 and August 2019 in women with a singleton pregnancy, by sonographers and fetal medicine doctors at the Maternity Ultrasound Unit, Oxford University Hospitals National Health Service (NHS) Foundation Trust, Oxfordshire, UK. In this center, all women are offered three routine ultrasound scans: first‐trimester crown–rump length dating12 at approximately 12 weeks' gestation, which includes nuchal translucency measurement for first‐trimester aneuploidy screening; a 20‐week anomaly scan; and a 36‐week growth scan in which estimated fetal weight (EFW) is computed13. Additionally, based on risk factors or clinical indications, women may be offered additional scans at other gestational ages14. Ultrasound examinations are carried out or supervised by accredited sonographers or fetal medicine doctors using standard ultrasound equipment. For quality control measures, the stored images and the reliability of measurements are regularly assessed using the INTERGROWTH‐21st quality control criteria15. Inclusion criteria were maternal age > 18 years of age and the ability to provide verbal and written informed consent in English.

andard ultrasound equipment. For quality control measures, the stored images and the reliability of measurements are regularly assessed using the INTERGROWTH‐21st quality control criteria15. Inclusion criteria were maternal age > 18 years of age and the ability to provide verbal and written informed consent in English. This study was part of a project entitled Perception Ultrasound by Learning Sonographic Experience (PULSE)16. This is an innovative interdisciplinary project designed to apply the latest ideas from machine learning and computer vision to build, from real‐world video data and other sensory data, computational models that describe how an expert sonographer performs a diagnostic study of a subject from multiple perceptual cues. By understanding closely how experts learn and undertake diagnostic ultrasound, we believe that we can build considerably more powerful assistive interpretation methods than have been possible so far. As part of the PULSE project, full‐length routine ultrasound scan videos are captured and recorded, probe movement is recorded and the point‐of‐gaze of the sonographer on the monitor of the ultrasound scanner is tracked.

n build considerably more powerful assistive interpretation methods than have been possible so far. As part of the PULSE project, full‐length routine ultrasound scan videos are captured and recorded, probe movement is recorded and the point‐of‐gaze of the sonographer on the monitor of the ultrasound scanner is tracked. All ultrasound scans included in this study were performed using commercial Voluson E8 version BT18 (GE Healthcare, Zipf, Austria) ultrasound machines, equipped with standard curvilinear (C2‐9‐D, C1‐5‐D) and three‐dimensional/four‐dimensional (RAB6‐D) probes. Synchronized eye tracking was undertaken using an eye tracker (Tobii Eye‐tracking Eye Tracker 4C, Danderyd, Sweden) attached to the ultrasound machine; the validity of eye‐tracking has been reported previously17. Of note, only one of the ultrasound machines in the center is equipped with eye‐tracking and recording devices, which limited the number of patients recruited during the study period. This study was approved by the UK Research Ethics Committee (Reference 18/WS/0051) and written informed consent was given by all participating pregnant women. Sonographers also consented to participate in the study at the outset but did not have any visual or other signal to know that the tracking devices were functioning during the examination. Funding for this study was granted by the European Research Council (ERC‐ADG‐2015 694 581, project PULSE) and the Engineering and Physical Sciences Research Council (EPSRC EP/M013774/1, project Seebibyte).

This study was approved by the UK Research Ethics Committee (Reference 18/WS/0051) and written informed consent was given by all participating pregnant women. Sonographers also consented to participate in the study at the outset but did not have any visual or other signal to know that the tracking devices were functioning during the examination. Funding for this study was granted by the European Research Council (ERC‐ADG‐2015 694 581, project PULSE) and the Engineering and Physical Sciences Research Council (EPSRC EP/M013774/1, project Seebibyte). Biometric measurement acquisition Acquisition of the three standard biometric measurements (i.e. HC, AC and FL) is a three‐stage process. First, the operator obtains an optimal acquisition of a standard biometric plane and freezes it on the screen. Next, the operator measures the biometric variable by placing calipers on the image; automatic caliper placement is turned on by default on the ultrasound machines used in our unit. The operator will often adjust caliper placement to achieve the best visual fit. During caliper placement and adjustment, ultrasound machines display on the screen a measurement box in which the measured length or circumference (in cm or mm) and the gestational age corresponding to the measurement (in weeks + days) are shown and updated in real‐time (Figure 1). Finally, the operator accepts the standard biometric plane measurement by saving the image with a visible measurement.

n a measurement box in which the measured length or circumference (in cm or mm) and the gestational age corresponding to the measurement (in weeks + days) are shown and updated in real‐time (Figure 1). Finally, the operator accepts the standard biometric plane measurement by saving the image with a visible measurement. Data extraction Each scan was automatically analyzed on a video frame‐by‐frame basis using a purpose‐built software program implemented in Python (http://www.python.org, version 3.7.0) using OpenCV (http://www.opencv.org, version 3.4) and Tesseract (http://www.github.com/tesseract-ocr, version 3.05). For each scan videoclip, the software program first detected the episodes of measuring a standard biometric plane by the appearance of the measurement box. Next, for each standard biometric measurement, the program detected uninterrupted fixations of the operator's eye on the measurement box lasting ≥ 100 ms, which is a widely accepted lower limit for eye fixation18. If eye fixation was interrupted, it was considered as one single episode of eye fixation if this interruption lasted ≤ 400 ms, or as a separate eye‐fixation episode if it lasted > 400 ms18, 19. Additionally, we verified the threshold for eye fixation by randomly looking at more than 50 detected fixations and ensuring that the threshold resulted in no false positives.

d as one single episode of eye fixation if this interruption lasted ≤ 400 ms, or as a separate eye‐fixation episode if it lasted > 400 ms18, 19. Additionally, we verified the threshold for eye fixation by randomly looking at more than 50 detected fixations and ensuring that the threshold resulted in no false positives. Concurrently, the software program stored the values displayed in the measurement box when the calipers were initially placed and when the operator accepted the measurement. Additionally, the software program stored the values displayed in the measurement box upon each detection of eye fixation on the measurement box. The measurement box values and parameters were extracted via optical character recognition. The Voluson E8 BT18 machine, by design, displays the observed (measured) gestational age as ‘OOR’ (out of range) in the measurement box when no standard curve is available for the measurement or the available curve does not cover the extremes of gestational age. In the current analysis, when this happened, the gestational‐age values were computed using the appropriate original formula. Expected value was defined as the gestational age at the time of the fetal growth scan, calculated based on the estimated due date that was established at the dating scan. Observed value was defined as the gestational age displayed in the measurement box which was based on the standard biometric measurement.

The Voluson E8 BT18 machine, by design, displays the observed (measured) gestational age as ‘OOR’ (out of range) in the measurement box when no standard curve is available for the measurement or the available curve does not cover the extremes of gestational age. In the current analysis, when this happened, the gestational‐age values were computed using the appropriate original formula. Expected value was defined as the gestational age at the time of the fetal growth scan, calculated based on the estimated due date that was established at the dating scan. Observed value was defined as the gestational age displayed in the measurement box which was based on the standard biometric measurement. Expected‐value bias was defined as occurring when the operator looked at the measurement box during the process of caliper adjustment before saving a standard biometric measurement (Figure 1 and Videoclip S1). After a specific standard biometric measurement (either HC, AC or FL) was saved, any additional same standard biometric measurement saved during the same examination was considered a repeat measurement.

Expected‐value bias was defined as occurring when the operator looked at the measurement box during the process of caliper adjustment before saving a standard biometric measurement (Figure 1 and Videoclip S1). After a specific standard biometric measurement (either HC, AC or FL) was saved, any additional same standard biometric measurement saved during the same examination was considered a repeat measurement. To evaluate the incidence of expected‐value bias we evaluated whether the operator looked at the measurement box before saving a standard biometric plane. To assess the impact of expected‐value bias, we: (I) measured how often the operators adjusted the calipers toward or away from the expected value; (II) evaluated the deviation of the observed from the expected values before and after the expected‐value bias took place (i.e. at the time the operator looked at the measurement box for the first time and when the measurement was saved); (III) compared the deviation between the observed and expected gestational age for standard biometric measurements that were repeated vs those that were not; and (IV) evaluated the impact of expected‐value bias on the EFW by calculating the lowest and highest possible EFW using the smallest and largest HC, AC, and FL measurements, respectively, before and after expected‐value bias occurred.

l age for standard biometric measurements that were repeated vs those that were not; and (IV) evaluated the impact of expected‐value bias on the EFW by calculating the lowest and highest possible EFW using the smallest and largest HC, AC, and FL measurements, respectively, before and after expected‐value bias occurred. Statistical analysis We report descriptive statistics. Continuous variables were compared using the Student's t‐test, Wilcoxon signed‐rank test (paired) or Mann–Whitney U‐test (unpaired). Comparison between saved (accepted) measurements and those recorded when the operator looked for the first time at the measurement box was investigated using multiple linear regression models. In order to evaluate independent relationships between the number of repeat measurements and the absolute deviation from the actual gestational age (expected value), we conducted a multifactor ANOVA analysis. Analyses were adjusted for the body mass index (BMI) of the pregnant woman and the number of years' scanning experience of the operator. P‐values < 0.05 were considered statistically significant. Analyses were carried out using R (http://www.r-project.org, version 3.5.2), Python (http://www.python.org, version 3.7.0), Pandas (http://pandas.pydata.org, version 0.24.0), SciPy (http://www.scipy.org, version 1.1.0) and Matplotlib (http://www.matplotlib.org, version 3.0.0).

ponds to a gestational age of 30 + 0 weeks, and ± 14.2 mm to a possible difference of 1.5 weeks, which was considered as just clinically acceptable. Similarly, for a measurement of 344 mm (the maximum HC, corresponding to 39 + 3 weeks) we would expect a possible difference of 4 weeks, larger than clinically acceptable. Performing similar calculations for BPD, there was a variation of ± 0.5 weeks for the minimum BPD and ± 2.5 weeks for the maximum BPD. The largest variation between examiners was seen for AC measurements. Those made by Examiner 2 differed from the measurements of Examiner 1 by ± 7% 95% of the time. Gestational age assessment in clinical practice Between 18 and 24 weeks (when biometry scans are used to assess gestational age if no first-trimester crown–rump length measurement is available), the variation in measurements was constant throughout the range of measurements (Figure 4); mean differences (95% CI) and limits of agreement of this subgroup were therefore calculated without log-transformation (Table 3). Figure 4 Bland–Altman plots of the interobserver differences in the measurement of biparietal diameter (BPD) (a,b), head circumference (HC) (c,d), abdominal circumference (AC) (e,f) and femur length (FL) (g,h) at 18–24 weeks' gestation, expressed as the measurement itself (a,c,e,g) and the corresponding estimated gestational age (b,d,f,h). For each graph, the solid line represents the mean difference and the dashed lines are the mean difference ± 2 SD (see Table 3).

were considered statistically significant. Analyses were carried out using R (http://www.r-project.org, version 3.5.2), Python (http://www.python.org, version 3.7.0), Pandas (http://pandas.pydata.org, version 0.24.0), SciPy (http://www.scipy.org, version 1.1.0) and Matplotlib (http://www.matplotlib.org, version 3.0.0). RESULTS During the study period, a total of 272 women undergoing a routine third‐trimester fetal growth scan were recruited. Demographic characteristics of the participants are displayed in Table 1. The mean gestational age at the time of the fetal growth scan was 34.6 ± 3.1 weeks. The examinations were performed by 16 operators, of which nine were accredited sonographers and seven fetal medicine doctors, with a median of 3 years' (range, 4 months to 14 years) clinical post‐accreditation experience in sonography (Table 2). Table 1 Characteristics of 272 women with singleton pregnancy included in study cohort Characteristic Value Maternal age (years) 31.9 ± 5.7 Smoker at booking 21 (7.7) BMI at < 15 weeks (kg/m2) 25.8 ± 5.3 Conception by IVF 4 (1.5) Nulliparous 123 (45.2) GA at fetal growth scan (weeks)* 34.6 ± 3.1 Pregnancy dating by CRL 249 (91.5) Pre‐eclampsia 7 (2.6) Gestational diabetes mellitus 11 (4.0) Preterm birth 11 (4.0) Vaginal birth 203 (74.6) Data are given as mean ± SD or n (%). * Gestational age (GA) based on estimated due date established at dating scan. BMI, body mass index; CRL, crown–rump length; IVF, in‐vitro fertilization. Table 2 Characteristics of 16 ultrasound operators who participated in study

Characteristic Value Maternal age (years) 31.9 ± 5.7 Smoker at booking 21 (7.7) BMI at < 15 weeks (kg/m2) 25.8 ± 5.3 Conception by IVF 4 (1.5) Nulliparous 123 (45.2) GA at fetal growth scan (weeks)* 34.6 ± 3.1 Pregnancy dating by CRL 249 (91.5) Pre‐eclampsia 7 (2.6) Gestational diabetes mellitus 11 (4.0) Preterm birth 11 (4.0) Vaginal birth 203 (74.6) Data are given as mean ± SD or n (%). * Gestational age (GA) based on estimated due date established at dating scan. BMI, body mass index; CRL, crown–rump length; IVF, in‐vitro fertilization. Table 2 Characteristics of 16 ultrasound operators who participated in study Characteristic Value Gender Female 14 (87.5) Male  2 (12.5) Clinical experience in scanning < 2 years  3 (18.8) 2–5 years  7 (43.8) 5–10 years  5 (31.3) > 10 years  1 (6.3) Accreditation Sonographer  9 (56.3) Fetal medicine doctor  7 (43.8) Data are given as n (%).

Table 2 Characteristics of 16 ultrasound operators who participated in study Characteristic Value Gender Female 14 (87.5) Male  2 (12.5) Clinical experience in scanning < 2 years  3 (18.8) 2–5 years  7 (43.8) 5–10 years  5 (31.3) > 10 years  1 (6.3) Accreditation Sonographer  9 (56.3) Fetal medicine doctor  7 (43.8) Data are given as n (%). A total of 1409 standard biometric plane measurements were made in the 272 scans, comprising 354 of the HC, 703 of the AC and 352 of the FL. We observed a risk of measurement bias in 91.4% of the measurements, of which 85.0%, 92.9% and 94.9% were of the HC, AC, and FL measurements, respectively (Table 3). Importantly, there was evidence that looking at the measurement box during caliper adjustment was likely due to bias rather than due to other reasons, as operators were more likely to adjust measurements towards the expected gestational age than to adjust it away from the expected gestational age (47.7% vs 19.7% overall; 49.5% vs 16.4% for HC; 51.5% vs 26.3% for AC; and 38.9% vs 9.6% for FL; P < 0.001 for all comparisons) (Table 3). Table 3 Number of measurements performed during fetal growth scan and incidence of expected‐value bias, according to standard biometric measurement Standard biometric measurement Saved measurements(n) Repeat measurements(n) Measurements per growth scan (mean ± SD) Biased measurements(%) Adjustment of measurement Towards expected GA* (%) Away from expected GA* (%) Mean adjustment towards expected GA* (days' gestation) P

Table 3 Number of measurements performed during fetal growth scan and incidence of expected‐value bias, according to standard biometric measurement Standard biometric measurement Saved measurements(n) Repeat measurements(n) Measurements per growth scan (mean ± SD) Biased measurements(%) Adjustment of measurement Towards expected GA* (%) Away from expected GA* (%) Mean adjustment towards expected GA* (days' gestation) P Head circumference  354  82 1.3 ± 0.6 85.0 49.5 16.4 2.3 ± 5.6  < 0.001 Abdominal circumference  703 431 2.6 ± 1.0 92.9 51.5 26.3 2.4 ± 10.4 < 0.001 Femur length  352  80 1.3 ± 0.7 94.9 38.9  9.6 3.2 ± 10.4 < 0.001 Total 1409 593 5.2 ± 1.7 91.4 47.7 19.7 2.6 ± 9.5  < 0.001 * Gestational age (GA) based on estimated due date established at dating scan. The risk of expected‐value bias applied to all operators, though it varied from 56% to 100% of measurements for the different operators. The correlation between years of scanning experience of an operator and the percent of measurements prone to bias was not statistically significant (P = 0.34).

Head circumference  354  82 1.3 ± 0.6 85.0 49.5 16.4 2.3 ± 5.6  < 0.001 Abdominal circumference  703 431 2.6 ± 1.0 92.9 51.5 26.3 2.4 ± 10.4 < 0.001 Femur length  352  80 1.3 ± 0.7 94.9 38.9  9.6 3.2 ± 10.4 < 0.001 Total 1409 593 5.2 ± 1.7 91.4 47.7 19.7 2.6 ± 9.5  < 0.001 * Gestational age (GA) based on estimated due date established at dating scan. The risk of expected‐value bias applied to all operators, though it varied from 56% to 100% of measurements for the different operators. The correlation between years of scanning experience of an operator and the percent of measurements prone to bias was not statistically significant (P = 0.34). The deviation of the observed gestational age (based on the biometric measurement) from the expected gestational age, expressed in days of gestation, before and after expected‐value bias occurred, is presented in Figure 2. We found a statistically significant difference in the mean observed gestational age before and after the operators looked at the measurement box, with the HC, AC and FL measurements being closer to the expected gestational age by 2.3 ± 5.6, 2.4 ± 10.4 and 3.2 ± 10.4 days of gestation, respectively (P < 0.001 for all comparisons). Additionally, we noted that values were closer to the mean after measurement bias occurred (reduction of the variance, Levene's test, P = 0.0255). These correlations remained statistically significant after multivariable analysis was performed, adjusting for maternal BMI and years' scanning experience of the operator as confounding variables. Additionally, when there was evidence of bias, we compared the measurement at the time the operator first looked at the measurement box and that eventually saved. We noted that the further the initial measurement was from the expected value, the larger was the adjustment of calipers toward the expected value (P < 0.001 for AC, HC and FL). This correlation remained significant after adjusting for operator experience and maternal BMI.

the measurement box and that eventually saved. We noted that the further the initial measurement was from the expected value, the larger was the adjustment of calipers toward the expected value (P < 0.001 for AC, HC and FL). This correlation remained significant after adjusting for operator experience and maternal BMI. Figure 2 Deviation of observed gestational age (GA), based on standard biometric measurement of head circumference (a), abdominal circumference (b) and femur length (c) at fetal growth scan, from expected GA, based on estimated due date established at dating scan, before (, ) and after (, ) expected‐value bias occurred, i.e. when operator first looked at measurement box and after measurement was saved.

urement of head circumference (a), abdominal circumference (b) and femur length (c) at fetal growth scan, from expected GA, based on estimated due date established at dating scan, before (, ) and after (, ) expected‐value bias occurred, i.e. when operator first looked at measurement box and after measurement was saved. UOG-21929-FIG-0002-bWe also compared the deviation of the observed from the expected gestational age between measurements that were repeated and those that were not. A total of 82, 431 and 80 measurements of HC, AC and FL, respectively, were repeated. Operators were more likely to repeat a measurement when this was far from the expected value. The observed gestational age was significantly closer to the expected gestational age for measurements that were not repeated. This means that the operators were more likely to acquire another image of the same standard biometric plane and measure again if the initial measurement was far from the expected value. The mean deviation of the observed from the expected gestational age for measurements that were repeated, compared with those that were not, was 15.1 ± 8.4 vs 10.2 ± 10.9 gestational days (P < 0.001) for HC measurements; 12.4 ± 14.3 vs 11.5 ± 12.3 days (P = 0.036) for AC measurements; and 13.3 ± 11.1 vs 7.7 ± 9.7 days (P < 0.001) for FL measurements (Figure 3). This correlation remained statistically significant after performing multivariable analysis, adjusting for maternal BMI and operator experience.

(P < 0.001) for HC measurements; 12.4 ± 14.3 vs 11.5 ± 12.3 days (P = 0.036) for AC measurements; and 13.3 ± 11.1 vs 7.7 ± 9.7 days (P < 0.001) for FL measurements (Figure 3). This correlation remained statistically significant after performing multivariable analysis, adjusting for maternal BMI and operator experience. Figure 3 Deviation of observed gestational age (GA), based on standard biometric measurement of head circumference (a), abdominal circumference (b) and femur length (c) at fetal growth scan, from expected GA, based on estimated due date established at dating scan, for measurements that were repeated (, ) and those that were not repeated (, ). UOG-21929-FIG-0003-bFinally, in order to estimate the impact of this potential expected‐value bias, we calculated the lowest and highest EFW, using respectively the smallest and largest biased HC, AC and FL measurements. The discordance, expressed in percentage terms, was 10.1% ± 6.5%. The Z‐score difference between the highest and lowest possible EFW was 0.83 ± 0.58. This means that 46 fetuses (17%; 95% CI, 12–21%) could be considered as small‐for‐gestational age, if using the smallest possible measurements, and appropriate‐for‐gestational age, if using the largest possible measurements. Similarly, in 34 scans (13%; 95% CI, 9–16%) the fetus could be considered as large‐for‐gestational age or appropriate‐for‐gestational age if the largest or smallest possible measurements, respectively, were used.

est possible measurements, and appropriate‐for‐gestational age, if using the largest possible measurements. Similarly, in 34 scans (13%; 95% CI, 9–16%) the fetus could be considered as large‐for‐gestational age or appropriate‐for‐gestational age if the largest or smallest possible measurements, respectively, were used. DISCUSSION This study has demonstrated that measurements undertaken during fetal growth scans are often biased by knowledge of the gestational age and the expected measurement for gestation. Operators tend to correct caliper placement at the time of the scan toward the expected measurement for the actual gestational age. The amount of correction correlates with the amount of deviation from the expected value. Additionally, we noted that operators were more likely to retake an image and repeat a measurement when the first measurement was far from the expected value. We did not find a correlation between the tendency to undertake such correction and the number of years' clinical experience or type of accreditation of the operator.

ionally, we noted that operators were more likely to retake an image and repeat a measurement when the first measurement was far from the expected value. We did not find a correlation between the tendency to undertake such correction and the number of years' clinical experience or type of accreditation of the operator. It is difficult to compare our findings with previous reports, as observer bias/expected‐value bias is not well studied in obstetric ultrasound. Nevertheless, unbiased and accurate measurement is a fundamental tenet of science. Such bias is not limited only to obstetric ultrasound, but can be encountered in many other medical fields, and is known to modify significantly clinical measurements as well as experimental results1. For example, in the case of blood pressure measurement, having an expectation of what it ought to be, might lead to an arbitrary adjustment of a non‐automatic reading20.

und, but can be encountered in many other medical fields, and is known to modify significantly clinical measurements as well as experimental results1. For example, in the case of blood pressure measurement, having an expectation of what it ought to be, might lead to an arbitrary adjustment of a non‐automatic reading20. The magnitude of the effect of expected‐value bias is difficult to ascertain and requires a study comparing blinded and non‐blinded fetal biometric measurements. Nevertheless, we found that the impact of bias on EFW may be as high as 10%, and that in 17% of scans the fetus could be considered as small‐for‐gestational age or appropriate‐for‐gestational age, depending on whether the smallest or the largest possible bias measurement was used. The corresponding figure for fetuses that could be considered as large‐for‐gestational age or appropriate‐for‐gestational age was 13%. This could lead to erroneous diagnosis of growth restriction, and thus to unnecessary intervention, maternal anxiety and iatrogenic perinatal morbidity, or it could result in classifying as normal a small‐for‐gestational‐age fetus, putting the pregnancy at risk for adverse perinatal outcome21. Hence, when making an obstetric decision, the possibility of bias in the estimation of fetal weight should also be taken into account. Moreover, in clinical practice, it is known that the detection rates for growth restriction during screening remain limited and one could hypothesize that expected‐value bias could be one of the reasons.

an obstetric decision, the possibility of bias in the estimation of fetal weight should also be taken into account. Moreover, in clinical practice, it is known that the detection rates for growth restriction during screening remain limited and one could hypothesize that expected‐value bias could be one of the reasons. Our findings also have obvious and important implications on research that is based on routine clinical data acquisition, for example when studying normal fetal growth. Bias in measurements means that any underlying formula programmed into the ultrasound system, relating gestational age to the fetal measurement, will have an important effect when aggregating data. It is for this reason that blinding operators to the measurement value is such a crucial step when creating normal ranges9, 10, 11. In addition, this study is part of the PULSE project, which is designed to apply the latest ideas from artificial intelligence, machine learning and computer vision to build computational models that describe how expert sonographers perform scanning. Our findings emphasize the importance of minimizing bias when training computer models to perform a task. This is because artificial intelligence is trained by humans who may introduce their own biases to the learning process, resulting in biased models. Based on current practice, algorithm training to measure standard biometric planes might result in a built‐in bias when automatically calculating fetal biometry. This bias can potentially even be amplified by the algorithm22, 23.

ns who may introduce their own biases to the learning process, resulting in biased models. Based on current practice, algorithm training to measure standard biometric planes might result in a built‐in bias when automatically calculating fetal biometry. This bias can potentially even be amplified by the algorithm22, 23. In our study all fetal growth scans were routine assessments and most fetuses were appropriate‐for‐gestational age. It is possible that this bias may be more pronounced in pregnancies with small‐ and large‐for‐gestational‐age fetuses, as greater measurement correction towards the expected value would be anticipated. This may be compounded by the well‐documented larger errors in fetal weight estimation in small‐ and large‐for‐gestational‐age fetuses24.

this bias may be more pronounced in pregnancies with small‐ and large‐for‐gestational‐age fetuses, as greater measurement correction towards the expected value would be anticipated. This may be compounded by the well‐documented larger errors in fetal weight estimation in small‐ and large‐for‐gestational‐age fetuses24. The accuracy and reliability of fetal biometry measurements are determined by the accuracy of standardized biometric plane acquisition25 and caliper placement. In this study, to evaluate the effect of bias during caliper placement, we tracked the eye movements of the operator, considering that risk of bias occurred when the operator looked at the measurement box while adjusting caliper placement or saving the image. However, a biased measurement does not necessarily mean that the measurement is incorrect. Extreme values are likely to represent a low‐quality acquisition rather than a fetal growth concern. Therefore, operators may commonly look at the displayed measurement to ensure that their measurement meets their expectation before adjusting the calipers. Likewise, adjusting the measurement away from the actual gestational age does not necessarily represent an unbiased measurement. For example, if the operator is aware of gestational diabetes, the operator may unconsciously perceive that the fetus is big, and hence measure it to be large‐for‐gestational age. Nevertheless, our findings suggest that, on average, operators adjust the measurement towards the expected measurement for gestational age. Similarly, performing a repeat standard biometric plane acquisition and measurement may represent good practice8. Nonetheless, operators may choose to acquire an additional standard biometric measurement due to an unsatisfactory self‐scoring quality assurance8 or because of a measurement value that does not match closely enough the expected one. We noticed measurements that were not repeated were closer to the expected value.

ce8. Nonetheless, operators may choose to acquire an additional standard biometric measurement due to an unsatisfactory self‐scoring quality assurance8 or because of a measurement value that does not match closely enough the expected one. We noticed measurements that were not repeated were closer to the expected value. Our study has some limitations. It was conducted in a single maternity unit which may not represent practice at other centers; nevertheless, we included 16 operators and the same finding was seen in all, making external validity more likely. In addition, even though the operators were aware that the scans and their eye movements were being recorded, they had not been informed of the aim of the current analysis meaning that it is unlikely that they acted differently while participating in this study. Another limitation is that the impact of expected‐value bias could only be estimated. To examine accurately the impact of bias would require performing a study in which operators are assigned randomly to blinding of measurements. However, the principle shown in this paper suggests that expected‐value bias is both common and clinically significant. We reported recently that operators rarely look at the safety indices while they scan26. This suggests that eye tracking of the operator is precise in detecting the point of gaze. The finding that operators look at measurements, but not bioeffects, is in accordance with our assumption. Finally, we used the actual gestational age as the reference (expected) value, however, in our setting this is based on a measurement performed at the dating scan, which may also be biased27.

ng the point of gaze. The finding that operators look at measurements, but not bioeffects, is in accordance with our assumption. Finally, we used the actual gestational age as the reference (expected) value, however, in our setting this is based on a measurement performed at the dating scan, which may also be biased27. In conclusion, observer bias towards expected values of fetal measurements is prevalent in routine third‐trimester growth scans. Further research should evaluate the added value of eliminating this bias to the overall accuracy of growth scans. To overcome it, ultrasound manufacturers should consider including settings that allow operators to be blinded before saving or ending ultrasound examinations. Supporting information Videoclip S1 Occurrence of expected‐value bias during measurement of abdominal circumference at 36 + 5 weeks' gestation. Initial observed (i.e. automatically calculated) gestational age is 38 + 6 weeks. After caliper adjustment, operator looks at measurement box that displays gestational age of 38 + 0 weeks. Then, operator adjusts caliper and looks again at measurement box. Final saved measurement equals gestational age of 37 + 2 weeks. Note that, in order to facilitate understanding of expected‐value bias, eye tracking is indicated on video by green dot; however, operator did not see this or any other indication of eye‐tracking function on screen during measurement. Click here for additional data file. Videoclip S2 Presentation of study at 29th ISUOG World Congress on Ultrasound in Obstetrics and Gynecology. Click here for additional data file.

Videoclip S1 Occurrence of expected‐value bias during measurement of abdominal circumference at 36 + 5 weeks' gestation. Initial observed (i.e. automatically calculated) gestational age is 38 + 6 weeks. After caliper adjustment, operator looks at measurement box that displays gestational age of 38 + 0 weeks. Then, operator adjusts caliper and looks again at measurement box. Final saved measurement equals gestational age of 37 + 2 weeks. Note that, in order to facilitate understanding of expected‐value bias, eye tracking is indicated on video by green dot; however, operator did not see this or any other indication of eye‐tracking function on screen during measurement. Click here for additional data file. Videoclip S2 Presentation of study at 29th ISUOG World Congress on Ultrasound in Obstetrics and Gynecology. Click here for additional data file. ACKNOWLEDGMENTS We are grateful to the sonographers and pregnant volunteers who participated in this study. A.T.P. is supported by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. This study was presented as an Oral Communication (OC19.02) at the 29th ISUOG World Congress on Ultrasound in Obstetrics and Gynecology (Videoclip S2).

Introduction Ultrasound examination of the fetus is a powerful tool for assessing gestational age, detecting multiple pregnancy and intrauterine growth restriction, and determining placental location1–5. Since the 1990s, almost every pregnant woman in developed countries has had access to between one and four routine scans during uncomplicated pregnancies6. However, in most developing countries antenatal ultrasound services are non-existent or inadequate. Those that are available are usually limited to tertiary centers or private hospitals in urban regions7–9. A lack of qualified sonographers and a shortage of ultrasound machines, most likely due to their high cost and maintenance difficulties, have been barriers to the implementation of routine ultrasound examination in many antenatal clinics in resource-poor settings. Recently, assistant medical officers, clinical officers, midwives or local radiographers have been identified as potential sonographers7, 10, 11. Given that ultrasound imaging has no value if the ultrasonographer is inadequately trained or inexperienced, recent efforts have concentrated on training8. Some African countries have reported promising results from starting ultrasound teaching programs10 but, as more developing countries introduce such programs, studies to ensure the quality and consistency of locally trained sonographers will be required.

or inexperienced, recent efforts have concentrated on training8. Some African countries have reported promising results from starting ultrasound teaching programs10 but, as more developing countries introduce such programs, studies to ensure the quality and consistency of locally trained sonographers will be required. At the Shoklo Malaria Research Unit (SMRU) local health workers (schooled until 16 years of age) have been trained in basic ultrasound imaging since 2001. They have performed approximately 3000 obstetric ultrasound scans per year in Maela refugee camp over the past 5 years. The aim of this study was to assess the intraobserver and interobserver agreement of fetal biometric measurements performed by these health workers.

een trained in basic ultrasound imaging since 2001. They have performed approximately 3000 obstetric ultrasound scans per year in Maela refugee camp over the past 5 years. The aim of this study was to assess the intraobserver and interobserver agreement of fetal biometric measurements performed by these health workers. Methods The SMRU is located on the Thai–Burmese border and has studied the epidemiology, prevention and treatment of malaria in pregnancy since 1986. It has five established clinics, one of which is based in refugee camp Maela, where Karen people (a minority group in Burma) are the principal inhabitants. In all its clinics the SMRU runs a program of antenatal care (ANC) to detect and treat all parasitemic episodes during pregnancy through weekly malaria screening in order to prevent maternal death12. Since the inception of this ANC program, all pregnant women have been encouraged to attend as early as possible in pregnancy. At the first visit (usually between 8 and 13 weeks' gestation), ultrasound imaging is used to determine viability, detect multiple pregnancy and estimate gestational age. A second scan is performed at 18–24 weeks to confirm gestation, viability and placental position. In women who do not have an early scan, gestational age assessment is based on fetal biometry scans between 18 and 24 weeks' gestation, or using the Dubowitz gestational age examination at birth if no such scan is available13.

cond scan is performed at 18–24 weeks to confirm gestation, viability and placental position. In women who do not have an early scan, gestational age assessment is based on fetal biometry scans between 18 and 24 weeks' gestation, or using the Dubowitz gestational age examination at birth if no such scan is available13. The SMRU introduced ultrasound examination for gestational age assessment owing to the low proportion of women who could reliably provide the date of their last menstrual period (LMP). In the past 3 years only 31% (994/3184) of women in Maela refugee camp provided a reliable LMP. When the ultrasound department in the antenatal clinic of the Maela refugee camp opened in 2001, one of the coauthors (S.L.M.D.), a local Karen health worker who was already skilled in Dubowitz assessment of gestational age, was trained in ultrasound gestational age assessment. A 3-month course of practical and theoretical training in obstetric ultrasound imaging was then developed (Figure 1) for newly employed staff, all of whom were chosen at interview on the basis of motivation, willingness to learn and proficiency in English. The course was based on World Health Organization (WHO) guidelines and British Medical Ultrasound Society (BMUS) recommendations14, 15. During the next 3 months all scans were verified by a senior sonographer. Only when the head of the department was satisfied with each person's scanning skills and written examination results were they permitted to scan alone. Figure 1 Photograph showing training in the ultrasound room at Maela refugee camp, 2008.

The SMRU introduced ultrasound examination for gestational age assessment owing to the low proportion of women who could reliably provide the date of their last menstrual period (LMP). In the past 3 years only 31% (994/3184) of women in Maela refugee camp provided a reliable LMP. When the ultrasound department in the antenatal clinic of the Maela refugee camp opened in 2001, one of the coauthors (S.L.M.D.), a local Karen health worker who was already skilled in Dubowitz assessment of gestational age, was trained in ultrasound gestational age assessment. A 3-month course of practical and theoretical training in obstetric ultrasound imaging was then developed (Figure 1) for newly employed staff, all of whom were chosen at interview on the basis of motivation, willingness to learn and proficiency in English. The course was based on World Health Organization (WHO) guidelines and British Medical Ultrasound Society (BMUS) recommendations14, 15. During the next 3 months all scans were verified by a senior sonographer. Only when the head of the department was satisfied with each person's scanning skills and written examination results were they permitted to scan alone. Figure 1 Photograph showing training in the ultrasound room at Maela refugee camp, 2008. As part of a larger fetal growth study, quality control evaluation (interobserver and intraobserver variability) was performed between four local sonographers and one expatriate doctor (M.J.R.), certified and experienced in obstetric ultrasound scanning. The Mahidol–Bangkok and Oxford University ethics committees approved both the main and quality control studies.

y control evaluation (interobserver and intraobserver variability) was performed between four local sonographers and one expatriate doctor (M.J.R.), certified and experienced in obstetric ultrasound scanning. The Mahidol–Bangkok and Oxford University ethics committees approved both the main and quality control studies. Every fifth pregnant woman attending the ANC was invited to participate in the study if she had an early (8–13-week) dating scan at the SMRU ANC, a singleton pregnancy, and a gestational age of between 16 and 40 weeks. A maximum of 15 women for each gestational week were invited. After obtaining written informed consent, an abdominal ultrasound examination was performed. At each examination, two examiners independently measured biparietal diameter (BPD), head circumference (HC), abdominal circumference (AC) and femur length (FL) in millimeters. Each image was acquired according to BMUS guidelines, ensuring that the image filled at least a third of the monitor screen. The machine automatically calculated the gestational age (weeks and days) from each measurement using Hadlock's charts16. Each examiner was blinded to his own results and the results of the other examiner. All measurements were obtained twice by one examiner (Examiner 1) to assess intraobserver variability, and once by another examiner (Examiner 2) to assess interobserver variability, resulting in 12 measurements per woman (i.e. three sets of four measurements).

d to his own results and the results of the other examiner. All measurements were obtained twice by one examiner (Examiner 1) to assess intraobserver variability, and once by another examiner (Examiner 2) to assess interobserver variability, resulting in 12 measurements per woman (i.e. three sets of four measurements). All scans were performed using a Toshiba Powervision 7000 machine (Toshiba, Tokyo, Japan) with a 3.75-MHz convex probe, which was donated by the University of Utrecht, The Netherlands. Owing to electrical surges in the refugee camp, a voltage stabilizer was used to operate the ultrasound scanner. Statistical analysis The extent to which measurements agree between two sonographers is limited by the amount of variation in repeated measurements made on the same subject by the same individual. This measurement error was assessed using the intraclass correlation coefficient (ICC), with a value of 1 (possible range 0 to 1) indicating no measurement error17. However, as the ICC will be artificially inflated owing to the large range of gestational ages included, summary measures (mean, minimum and maximum differences and SD) for each sonographer are also reported.

ass correlation coefficient (ICC), with a value of 1 (possible range 0 to 1) indicating no measurement error17. However, as the ICC will be artificially inflated owing to the large range of gestational ages included, summary measures (mean, minimum and maximum differences and SD) for each sonographer are also reported. The agreement between the mean of the two measurements made by Examiner 1 and the measurement made by Examiner 2 was then estimated, using the 95% limits of agreement method proposed by Bland and Altman18, 19. Data were initially plotted, with a line of equality, to gauge the degree of agreement between measurements. All points would lie on the line of equality if the two examiners reported exactly the same measurements. The assumptions that the SD of repeated measures was not related to the magnitude of the ultrasound measurements and that the differences between the measurements followed a normal distribution were then checked visually using scatter plots and histograms, respectively. If the assumptions were not met, calculations were carried out on log-transformed values and the antilog was taken to obtain limits of agreement that could be related to the original scale of measurement18, 20. As gestational age assessment in clinical practice normally occurs between 18 and 24 weeks, interobserver variation was calculated for this subgroup of measurements, as well as for the entire dataset.

d the antilog was taken to obtain limits of agreement that could be related to the original scale of measurement18, 20. As gestational age assessment in clinical practice normally occurs between 18 and 24 weeks, interobserver variation was calculated for this subgroup of measurements, as well as for the entire dataset. Biometry measurements in the second trimester have an accuracy of ± 1 week in estimating gestational age, whereas the accuracy decreases to ± 2 weeks in the third trimester21–25. A difference in measurements (millimeters) between the two examiners that corresponded to a difference in gestational age of ± 1 week or less was considered to be very good agreement21–25. Clinically acceptable and unacceptable findings were gestational age differences that were ± 1–2 weeks and more than ± 2 weeks, respectively. We used the mean of the repeated measurements taken by Examiner 1 against the measurements made by Examiner 2 to assess interobserver variation. One could expect the estimate of SD to be smaller (owing to removal of repeated measurement error) by using the mean18. When compared with using only one set of measurements from Examiner 1, however, the results differed by less than 1 mm for all parameters, regardless of whether the first or second set of repeated measurements was used (results not shown). Therefore, only the results using the mean of Examiner 1's measurements are reported here, and no adjustments to the SD were made. All analyses were carried out using STATA/SE, version 9.2 for Windows (StataCorp LP, College Station, Texas, USA).

We used the mean of the repeated measurements taken by Examiner 1 against the measurements made by Examiner 2 to assess interobserver variation. One could expect the estimate of SD to be smaller (owing to removal of repeated measurement error) by using the mean18. When compared with using only one set of measurements from Examiner 1, however, the results differed by less than 1 mm for all parameters, regardless of whether the first or second set of repeated measurements was used (results not shown). Therefore, only the results using the mean of Examiner 1's measurements are reported here, and no adjustments to the SD were made. All analyses were carried out using STATA/SE, version 9.2 for Windows (StataCorp LP, College Station, Texas, USA). Results Between April and September 2008, 349 pregnant women consented to the ultrasound examination. The median gestational age was 27 (range, 16–40) weeks. It was possible to complete the examination and obtain all 12 measurements in all women, and so a total of 4188 measurements were obtained. Education level of local health workers All four local health workers involved in the obstetric ultrasound course agreed to participate in this quality control study. One had completed 3 years of training as a nurse at a recognized institution in Burma. The others did not have any tertiary education but had completed school to grade 10 (16 years old). At the start of the study, they had a median of 20 (range, 12–62) months of work experience.

rticipate in this quality control study. One had completed 3 years of training as a nurse at a recognized institution in Burma. The others did not have any tertiary education but had completed school to grade 10 (16 years old). At the start of the study, they had a median of 20 (range, 12–62) months of work experience. Intraobserver variation Table 1 shows the summary of the repeated measurements of all examiners. The ICC for all four parameters (BPD, HC, AC, FL) was greater than 0.99 for all four trainees and the doctor (range 0.996–0.998), indicating that almost all of the variation observed was due to differences between patients rather than differences in the repeated measurements taken by an examiner on any one patient. Table 1 Mean, minimum and maximum differences for each pair of measurements obtained by the same locally trained sonographer (A–D) or the doctor BPD (mm) HC (mm) AC (mm) FL (mm)

Intraobserver variation Table 1 shows the summary of the repeated measurements of all examiners. The ICC for all four parameters (BPD, HC, AC, FL) was greater than 0.99 for all four trainees and the doctor (range 0.996–0.998), indicating that almost all of the variation observed was due to differences between patients rather than differences in the repeated measurements taken by an examiner on any one patient. Table 1 Mean, minimum and maximum differences for each pair of measurements obtained by the same locally trained sonographer (A–D) or the doctor BPD (mm) HC (mm) AC (mm) FL (mm) Examiner n Mean difference (SD) Min, max Mean difference (SD) Min, max Mean difference (SD) Min, max Mean difference (SD) Min, max A 157 0.15 (1.62) − 5.5, 7.7 0.43 (6.52) − 26, 20 0.01 (8.54) − 24, 36 − 0.28 (1.56) − 5.4, 4.3 B 70 − 0.11 (1.05) − 2.3, 4.9 0.90 (4.86) − 14, 16 0.51 (5.90) − 14, 24 0.03 (1.27) − 3.3, 5.1 C 67 − 0.26 (1.43) − 3.7, 2.9 − 0.64 (6.62) − 22, 15 − 0.93 (7.97) − 33, 14 0.14 (1.59) − 3.6, 8.7 D 18 0.08 (1.46) − 2.0, 3.3 − 1.44 (4.51) − 9, 5 0.39 (10.6) − 30, 27 0.21 (1.53) − 2.5, 3.2 Doctor 37 0.05 (1.16) − 3.2, 3.3 1.27 (4.56) − 6, 19 0.41 (5.45) − 13, 12 − 0.02 (0.91) − 1.9, 2.1 AC, abdominal circumference; BPD, biparietal diameter; FL, femur length; HC, head circumference; max, maximum difference; min, minimum difference; n, number of pairs of measurements obtained by each examiner.

) − 2.5, 3.2 Doctor 37 0.05 (1.16) − 3.2, 3.3 1.27 (4.56) − 6, 19 0.41 (5.45) − 13, 12 − 0.02 (0.91) − 1.9, 2.1 AC, abdominal circumference; BPD, biparietal diameter; FL, femur length; HC, head circumference; max, maximum difference; min, minimum difference; n, number of pairs of measurements obtained by each examiner. Interobserver variation The agreement between the mean of the two measurements made by Examiner 1 was compared with the single measurement made by Examiner 2 to assess interobserver variation. This was done for the complete dataset (Table 2) as well as for the subset of measurements obtained for pregnancies at 18–24 weeks (Table 3). The distribution of mean differences was approximately normal for each of the four parameters but the SDs and differences for BPD, HC and AC increased with the magnitude of the measurements (Figures 2 and 3). These parameters were log-transformed for analysis and the back-transformed values used to estimate the ‘V-shaped’ 95% limits of agreement (i.e. the range within which measurements were expected to agree 95% of the time increased as the size of the measurement increased)18, 20. Figure 2 Scatter plots of SD against the average of three measurements for biparietal diameter (BPD) (a), head circumference (HC) (b), abdominal circumference (AC) (c) and femur length (FL) (d) in each fetus. For each graph, the solid line represents the regression line.

Interobserver variation The agreement between the mean of the two measurements made by Examiner 1 was compared with the single measurement made by Examiner 2 to assess interobserver variation. This was done for the complete dataset (Table 2) as well as for the subset of measurements obtained for pregnancies at 18–24 weeks (Table 3). The distribution of mean differences was approximately normal for each of the four parameters but the SDs and differences for BPD, HC and AC increased with the magnitude of the measurements (Figures 2 and 3). These parameters were log-transformed for analysis and the back-transformed values used to estimate the ‘V-shaped’ 95% limits of agreement (i.e. the range within which measurements were expected to agree 95% of the time increased as the size of the measurement increased)18, 20. Figure 2 Scatter plots of SD against the average of three measurements for biparietal diameter (BPD) (a), head circumference (HC) (b), abdominal circumference (AC) (c) and femur length (FL) (d) in each fetus. For each graph, the solid line represents the regression line. Figure 3 Bland–Altman plots of the interobserver differences in the measurement of biparietal diameter (BPD) (a), head circumference (HC) (b), abdominal circumference (AC) (c) and femur length (FL) (d), showing that the variation increased with the magnitude of the measurements for BPD, HC and AC. For each graph, the solid line represents the mean difference and the dashed lines are the mean difference ± 2 SD.

diameter (BPD) (a), head circumference (HC) (b), abdominal circumference (AC) (c) and femur length (FL) (d), showing that the variation increased with the magnitude of the measurements for BPD, HC and AC. For each graph, the solid line represents the mean difference and the dashed lines are the mean difference ± 2 SD. Table 2 Mean difference in ultrasound measurements obtained by two different examiners on 349 women at 16–40 weeks' gestation Parameter n Mean difference (mm (95% CI)) BPD 349 − 0.12 (−0.30 to 0.06) HC 349 − 0.11 (−0.77 to 0.54) AC 349 − 0.09 (−0.98 to 0.80) FL 349 − 0.13 (−0.28 to 0.03) AC, abdominal circumference; BPD, biparietal diameter; FL, femur length; HC, head circumference. Table 3 Mean difference and 95% limits of agreement (LOA) by measurement and corresponding estimated gestational age of ultrasound measurements obtained by two different examiners on 90 fetuses at 18–24 weeks' gestation Measurement Gestational age Parameter n Mean difference (mm (95% CI)) SD (mm) 95% LOA (mm) Mean difference (weeks (95% CI)) SD (weeks) BPD 90 − 0.43 (−0.68 to − 0.17) 1.21 − 2.80 to 1.94 − 0.12 (−0.19 to − 0.04) 0.36 HC 90 − 1.63 (−2.63 to − 0.62) 4.80 − 11.0 to 7.77 − 0.12 (−0.20 to − 0.03) 0.42 AC 90 − 0.59 (−1.77 to 0.60) 5.65 − 11.7 to 10.45 − 0.03 (−0.13 to 0.07) 0.50 FL 90 − 0.35 (−0.64 to − 0.07) 1.37 − 3.03 to 2.33 − 0.11 (−0.21 to 0) 0.50 AC, abdominal circumference; BPD, biparietal diameter; FL, femur length; HC, head circumference.

ominal circumference (AC) (e,f) and femur length (FL) (g,h) at 18–24 weeks' gestation, expressed as the measurement itself (a,c,e,g) and the corresponding estimated gestational age (b,d,f,h). For each graph, the solid line represents the mean difference and the dashed lines are the mean difference ± 2 SD (see Table 3). The largest mean difference was for HC measurements (Table 3 and Figure 4), indicating that the measurements by Examiner 2 differed from those made by the first examiner by 1.63 mm (95% CI, 0.62–2.63 mm). The 95% limits of agreement indicated that the measurements made by Examiner 2 could be expected to be within 11.0 mm lower to 7.8 mm higher than the measurements made by Examiner 1, 95% of the time (Table 3). This corresponds to a possible difference in estimation of gestational age of less than ± 1 week. Similarly, differences of less than ± 1 week were estimated for BPD, AC and FL. Comparison between the doctor and local trainees The expatriate doctor took at least one set of measurements on 124 women. Scatter plots between his measurements and those made by the trained health workers showed that all points were tightly clustered around the line of equality for all four parameters (Figure 5), indicating a high degree of agreement in ultrasound use by both teacher and students. Figure 5 Scatter plots of fetal biometry measurements (n = 124) made by the students against those made by the doctor for biparietal diameter (BPD) (a), head circumference (HC) (b), abdominal circumference (AC) (c) and femur length (FL) (d), with the line of equality shown for each.

Comparison between the doctor and local trainees The expatriate doctor took at least one set of measurements on 124 women. Scatter plots between his measurements and those made by the trained health workers showed that all points were tightly clustered around the line of equality for all four parameters (Figure 5), indicating a high degree of agreement in ultrasound use by both teacher and students. Figure 5 Scatter plots of fetal biometry measurements (n = 124) made by the students against those made by the doctor for biparietal diameter (BPD) (a), head circumference (HC) (b), abdominal circumference (AC) (c) and femur length (FL) (d), with the line of equality shown for each. Discussion In this study we found that local health workers can be trained to use ultrasound imaging reliably and consistently to assess gestational age. The intraobserver variation (ICC all > 0.99) demonstrated that measurements are made consistently by the same sonographer. For gestational ages between 18 and 24 weeks, there was a difference of less than 1 week in gestational age estimated using the measurements made by different examiners. In addition, when compared with the skill of an experienced doctor, the local trainees demonstrated a high level of agreement in measuring all four parameters. These findings reassured us that the criteria for selecting the trainees were adequate.

tional age estimated using the measurements made by different examiners. In addition, when compared with the skill of an experienced doctor, the local trainees demonstrated a high level of agreement in measuring all four parameters. These findings reassured us that the criteria for selecting the trainees were adequate. In developed countries it has long been established that fetal biometry at between 14 and 22 weeks' gestation can accurately predict gestational age within ± 7 days (± 2 SD)21–25. The variation in measurements between examiners in this study falls within this period. Our overall findings, therefore, strengthen the argument that obstetric ultrasound imaging can be introduced in developing countries for gestational age assessment, particularly when one considers the unreliability of LMP recall in such settings13, 26. It is essential to maintain quality control in any antenatal ultrasound service to ensure that the data obtained are clinically meaningful, e.g. by estimating the accuracy and reproducibility of the fetal biometry measurements taken by sonographers18, 27–29. In SMRU clinics, quality control is achieved by routinely taking all ultrasound measurements twice to assess intraobserver variability, and by an expatriate doctor qualified in ultrasound imaging annually checking the skills of all sonographers. Reassuringly, therefore, the measurement errors for gestational age estimation in this study were comparable to those obtained by highly experienced sonographers29.

ts twice to assess intraobserver variability, and by an expatriate doctor qualified in ultrasound imaging annually checking the skills of all sonographers. Reassuringly, therefore, the measurement errors for gestational age estimation in this study were comparable to those obtained by highly experienced sonographers29. Every measurement in clinical science is associated with error and, unsurprisingly, the variation increased as BPD, HC and AC sizes increased. This was not the case for FL, perhaps because the clearly defined landmarks of the FL (two edges of the femur bone, which are not affected by fetal breathing as in AC measurements) might have contributed to reducing the variation between measurements. Apart from trained health workers, robust ultrasound machines are needed to make obstetric ultrasound imaging available in remote areas. Unfortunately, as observed by Kurjak and Breyer, ‘many developing countries cannot afford to buy good quality ultrasound diagnostic instruments and do not have enough trained specialists who can devote a large fraction of their active time to the science and art of ultrasound diagnosis’7. However, ultrasound imaging has become more feasible in developing countries as machines become less expensive and require less servicing6, 10, 30, 31.

diagnostic instruments and do not have enough trained specialists who can devote a large fraction of their active time to the science and art of ultrasound diagnosis’7. However, ultrasound imaging has become more feasible in developing countries as machines become less expensive and require less servicing6, 10, 30, 31. To solve the problem of the lack of trained sonographers, the WHO has recognized the urgent need to raise education levels in ultrasound scanning in developing nations32. Hence, in 1998, it published a report concerning the essentials, principles and standards of training for both physicians and allied health professionals in diagnostic ultrasound imaging14. In reality, however, physicians in developing countries are heavily overloaded with work, resulting in inadequate use of available ultrasound machines7, 8. Thus, other health workers have been identified as potential sonographers7, 10, 11. Several reports of international ultrasound training programs have been published previously10, 32–34. Some of these were based in district hospitals in developing countries, where local health workers were trained successfully in theoretical and practical scanning skills. In this study, we have shown that candidates with limited or no tertiary education and limited English in a refugee camp can acquire good quality basic ultrasound skills for gestational age estimation with a short training course, a period of on-the-job training and ongoing quality control measures.

al scanning skills. In this study, we have shown that candidates with limited or no tertiary education and limited English in a refugee camp can acquire good quality basic ultrasound skills for gestational age estimation with a short training course, a period of on-the-job training and ongoing quality control measures. To our knowledge, this is the first report on quality assurance of gestational age estimation of locally trained sonographers in a refugee camp. Our results show that adequately trained health workers, working in a well organized unit with ongoing quality control, can obtain accurate fetal biometry measurements, whether the scan is performed by the same sonographer or by different sonographers. Given the importance of gestational age assessment in obstetric management, we recommend that ultrasound machines are made available and that ultrasound training is provided for local health workers in developing countries or resource-poor settings. We thank all the staff of the SMRU, especially Ei Pet, Hser Gay Wah, Ketkeaw Kittirawi and Suporn Kiricharoen for the ultrasound scanning, Moo Kho Paw and the midwives for the antenatal care of pregnant women and recording of ultrasound measurements, and Khun Tip and his group for logistical support. We thank Ph. Stoutenbeek and the Department of Obstetrics of the University Medical Center Utrecht, The Netherlands for donation of the ultrasound scanner. A.T.P. was funded by the NIHR Biomedical Research Programme. None of the authors had a conflict of interest.

INTRODUCTION The safe and effective delivery of a cancer‐screening program depends on the provision of a high‐quality service, continued development of the personnel delivering and running the service and a systematic approach linking all the activities involved in the identification of the cancer1. To monitor and evaluate these processes, quality assurance (QA) programs are essential. They are major contributors to the success of the breast and cervical screening programs in the UK. QA processes are often developed during the course of screening trials to facilitate future clinical implementation if appropriate. This is especially relevant to screening tests with a significant subjective element such as transvaginal ultrasound (TVS), which is integral to screening strategies for ovarian cancer2. In the last decade, four major ovarian cancer screening trials with different screening protocols have evaluated the efficacy of TVS as a screening test, with variable results3, 4, 5, 6. The single‐center Kentucky Ovarian Cancer Ultrasound Screening study reported a possible survival benefit3, whereas screening using a combination of TVS and CA 125 did not result in a stage shift in the Japanese Shizuoka Cohort study4 or demonstrate a mortality benefit in the USA Prostate, Lung, Colorectal and Ovarian (PLCO) screening trial5. The mortality impact of screening in the United Kingdom Collaborative Trial of Ovarian Cancer Screening (UKCTOCS)6, which completed screening in 2011, is awaited.

ge shift in the Japanese Shizuoka Cohort study4 or demonstrate a mortality benefit in the USA Prostate, Lung, Colorectal and Ovarian (PLCO) screening trial5. The mortality impact of screening in the United Kingdom Collaborative Trial of Ovarian Cancer Screening (UKCTOCS)6, which completed screening in 2011, is awaited. One of the concerns regarding the use of ultrasound in screening has been the considerable interobserver variability reported in ovarian visualization in large screening studies, and the fact that the accuracy of interpretation depends on the operator's experience2. Here we describe the methods used to ensure delivery of high‐quality ultrasound scanning in a multicenter setting, to define parameters for QA during TVS examination of postmenopausal ovaries and to assess the impact of QA processes over time on visualization rates (VR) of the ovary for individual sonographers and regional centers, after adjustment for non‐subjective factors that can impact on VR2. METHODS Participants, design and follow‐up within the UKCTOCS

One of the concerns regarding the use of ultrasound in screening has been the considerable interobserver variability reported in ovarian visualization in large screening studies, and the fact that the accuracy of interpretation depends on the operator's experience2. Here we describe the methods used to ensure delivery of high‐quality ultrasound scanning in a multicenter setting, to define parameters for QA during TVS examination of postmenopausal ovaries and to assess the impact of QA processes over time on visualization rates (VR) of the ovary for individual sonographers and regional centers, after adjustment for non‐subjective factors that can impact on VR2. METHODS Participants, design and follow‐up within the UKCTOCS Details regarding volunteers and design of the trial have been described previously in greater detail6, 7. In brief, of the 202 638 women recruited to the trial, 50 639 were randomized to the ultrasound arm and underwent annual TVS screening between October 2001 and December 2011. Transabdominal ultrasound (TAS) was undertaken only when TVS was declined by the participant. The examinations were performed by sonographers at 13 participating regional centers. All scan data were entered contemporaneously on a central web‐based trial management system. Women were followed up for a diagnosis of cancer through a ‘flagging study’ via the Health and Social Care Information Center (formerly the UK Office of National Statistics) and through postal questionnaire sent 3½ years after the women were randomized to the trial. This analysis was restricted to all women in the ultrasound group who had a scan by 31 December 2010. The UKCTOCS study was approved by the UK North West Multicentre Research Ethics Committees (North West MREC 00/8/34). It is registered as an International Standard Randomized Controlled Trial, number ISRCTN22488978.

l. This analysis was restricted to all women in the ultrasound group who had a scan by 31 December 2010. The UKCTOCS study was approved by the UK North West Multicentre Research Ethics Committees (North West MREC 00/8/34). It is registered as an International Standard Randomized Controlled Trial, number ISRCTN22488978. Ultrasound protocol The ultrasound protocol addressed all aspects of scanning during the trial. It included the standard operating procedure (SOP) on scanning the pelvis in postmenopausal women, which provided detailed guidance on the correct techniques to accurately examine the pelvis (starting with the uterus in the sagittal and transverse sections, reducing the depth and following the ovarian ligament out to the pelvic sidewall) and to identify the ovary based on its ultrasound appearance (small static hypoechoic area between the iliac vessel and ovarian ligament, adjacent to the iliac vessel anywhere along its length, and usually in close proximity to the uterus). Measurement of three ovarian diameters in two planes for the automated calculation of ovarian volume by the trial management system was detailed, as were suggestions for image optimization (abdominal palpation to displace the bowel, using the dual screen, manipulating the image to view the ovary in the middle of the screen, increasing the depth or using zoom to magnify the image of the ovary, positioning focus correctly, using frequency to improve resolution at required depth, slow frame rates and using the scroll facility to obtain the best frame after freezing). The classification and reporting of ovarian/adnexal lesions were standardized. Algorithms for the management of the findings are detailed elsewhere6.

tioning focus correctly, using frequency to improve resolution at required depth, slow frame rates and using the scroll facility to obtain the best frame after freezing). The classification and reporting of ovarian/adnexal lesions were standardized. Algorithms for the management of the findings are detailed elsewhere6. If the sonographer, the trial center team or the center's lead consultant had concerns regarding an ultrasound examination, they could request a review of the images by a senior UKCTOCS investigator with ultrasound expertise at the coordinating center. Ultrasound machines Dedicated centrally procured ultrasound machines with service contracts were used across the regional centers. At all centers, the Kretz SA9900 ultrasound machine (Medison, Seoul, South Korea) was used between 2001 and 2007, followed by the Medison Accuvix ultrasound machine from 2007 to 2011. At the beginning of the trial, and when ultrasound machines were upgraded, centers were visited to ensure uniform settings were used on all machines. The ultrasound machines were calibrated and maintained regularly under a centrally agreed UKCTOCS service contract. Key components of the QA process Ultrasound subcommittee

Ultrasound machines Dedicated centrally procured ultrasound machines with service contracts were used across the regional centers. At all centers, the Kretz SA9900 ultrasound machine (Medison, Seoul, South Korea) was used between 2001 and 2007, followed by the Medison Accuvix ultrasound machine from 2007 to 2011. At the beginning of the trial, and when ultrasound machines were upgraded, centers were visited to ensure uniform settings were used on all machines. The ultrasound machines were calibrated and maintained regularly under a centrally agreed UKCTOCS service contract. Key components of the QA process Ultrasound subcommittee Implementation of the ultrasound arm and set up and running of the QA processes were overseen initially by the team at the coordinating center and the trial investigators with ultrasound expertise. In 2006, a formal ultrasound subcommittee was formed. Chaired by the trial coordinator, the committee included senior trial gynecologists with expertise in gynecological imaging, radiologists experienced in abdominal and pelvic imaging, a national lead sonographer (NLS) for the trial, senior sonographers from regional centers and the coordinating center team, who were involved in day‐to‐day monitoring of the ultrasound protocol and QA process. The subcommittee managed the training and set up an accreditation process for sonographers, monitoring of adherence to the ultrasound protocol, fail‐safe monitoring and further development of the QA processes. They also had oversight of the logistics involved in implementation of the ultrasound arm of the trial. The subcommittee met three times a year with e‐mail updates from the NLS on a monthly basis.

hers, monitoring of adherence to the ultrasound protocol, fail‐safe monitoring and further development of the QA processes. They also had oversight of the logistics involved in implementation of the ultrasound arm of the trial. The subcommittee met three times a year with e‐mail updates from the NLS on a monthly basis. National lead sonographer (NLS) The NLS was a senior sonographer (superintendent level) with extensive experience in gynecological scanning and management of sonographers. She was appointed to the coordinating center team to oversee the day‐to‐day delivery of ultrasound across regional centers and take on QA management and execution of the ultrasound subcommittee's recommendations. Central to this were assessing and addressing the training needs of each sonographer, developing a system for their accreditation and re‐accreditation, running weekly fail‐safe checks on scan reports to monitor discrepancies or errors and implementing QA monitoring. In addition, the NLS worked with the regional centers to manage the logistics of delivering an average of 3000 scans per center per year following the end of recruitment. In her absence, the clinical research fellow ran the fail‐safe checks and QA monitoring. Personnel scanning in trial

The NLS was a senior sonographer (superintendent level) with extensive experience in gynecological scanning and management of sonographers. She was appointed to the coordinating center team to oversee the day‐to‐day delivery of ultrasound across regional centers and take on QA management and execution of the ultrasound subcommittee's recommendations. Central to this were assessing and addressing the training needs of each sonographer, developing a system for their accreditation and re‐accreditation, running weekly fail‐safe checks on scan reports to monitor discrepancies or errors and implementing QA monitoring. In addition, the NLS worked with the regional centers to manage the logistics of delivering an average of 3000 scans per center per year following the end of recruitment. In her absence, the clinical research fellow ran the fail‐safe checks and QA monitoring. Personnel scanning in trial Type 1 sonographers (certified sonographers, trained midwives or doctors in the National Health Service (NHS) trained in gynecological scanning) performed TVS as a first line‐test (level 1) in the ultrasound arm6. If the results of the level 1 scan were normal, the women were returned to annual screening and scanned the following year by a type 1 sonographer. However, on detection of an abnormality on the level 1 scan, a repeat scan (level 2) was arranged6. These were performed by type 2 sonographers (senior sonographers, usually at superintendent level, experienced gynecologists or radiologists) with expertise in gynecological scanning6. Induction of sonographers

Type 1 sonographers (certified sonographers, trained midwives or doctors in the National Health Service (NHS) trained in gynecological scanning) performed TVS as a first line‐test (level 1) in the ultrasound arm6. If the results of the level 1 scan were normal, the women were returned to annual screening and scanned the following year by a type 1 sonographer. However, on detection of an abnormality on the level 1 scan, a repeat scan (level 2) was arranged6. These were performed by type 2 sonographers (senior sonographers, usually at superintendent level, experienced gynecologists or radiologists) with expertise in gynecological scanning6. Induction of sonographers Each sonographer commencing scanning in the trial was required to submit their curriculum vitae giving details of their qualifications and scanning experience. They received a current copy of the scanning protocol. On‐site training was provided to familiarize each with the protocol and recommended methods of scanning before scanning independently on the trial. The NLS, designated senior trial gynecologists or radiologists with ultrasound expertise or, occasionally, a delegated experienced local sonographer, supervised the new sonographer for a minimum of two scanning sessions, assessed competency and then authorized the sonographer to scan independently in the trial. A practical assessment was arranged 3 months after the sonographers joined the trial and this was later incorporated into the accreditation process. Ongoing training of sonographers

Each sonographer commencing scanning in the trial was required to submit their curriculum vitae giving details of their qualifications and scanning experience. They received a current copy of the scanning protocol. On‐site training was provided to familiarize each with the protocol and recommended methods of scanning before scanning independently on the trial. The NLS, designated senior trial gynecologists or radiologists with ultrasound expertise or, occasionally, a delegated experienced local sonographer, supervised the new sonographer for a minimum of two scanning sessions, assessed competency and then authorized the sonographer to scan independently in the trial. A practical assessment was arranged 3 months after the sonographers joined the trial and this was later incorporated into the accreditation process. Ongoing training of sonographers Central training days, often spanning weekends, were organized twice a year for sonographers, nurses and other members of the regional center teams. This training included talks on recommended and standardized scanning techniques, definitions of key terms, quality checks and the ultrasound protocol. A mandatory component was a 2‐h case‐based discussion session that included both screen‐detected and screen‐missed (interval) cancers with an emphasis on the availability of a second opinion when faced with equivocal scan findings. It was obligatory for all sonographers to attend at least one study day per year. In addition, all centers were visited at the start, and on a regular basis during the course of the trial, by senior investigators with ultrasound expertise, accompanied by the clinical fellow or NLS. These visits included formal practical training sessions, talks to reinforce key messages and one‐to‐one scanning and assessment, especially of those identified on QA monitoring to have ovarian VRs below 60%.

rse of the trial, by senior investigators with ultrasound expertise, accompanied by the clinical fellow or NLS. These visits included formal practical training sessions, talks to reinforce key messages and one‐to‐one scanning and assessment, especially of those identified on QA monitoring to have ovarian VRs below 60%. The coordinating center also circulated newsletters to the sonographers and regional center nurses, communicating updates and news from the regional centers. Accreditation for individual sonographers was initiated in 2008 and comprised completion of a questionnaire to demonstrate understanding and knowledge of the UKCTOCS protocol, assessment of VR by review of scan data over a 3‐month period, practical assessment by the NLS, which included scanning of a minimum of five volunteers, and submission of nine sets of images for central review by the NLS and a subcommittee member with expertise in pelvic scanning. Ultrasound data collection and data entry Scan data regarding visualization of the ovaries, reasons for non‐visualization of the ovaries, morphological findings of the ovary, details of any abnormality seen, endometrial thickness measurements, presence of free fluid in the pouch of Douglas and any other abnormalities were recorded on a standard data form. In addition, the overall pictorial impression of abnormal morphology was collected initially using the Kentucky pictorial representation of morphology index and subsequently, from 2004, using the International Ovarian Tumor Analysis Group pictorial classification8, 9.

bnormalities were recorded on a standard data form. In addition, the overall pictorial impression of abnormal morphology was collected initially using the Kentucky pictorial representation of morphology index and subsequently, from 2004, using the International Ovarian Tumor Analysis Group pictorial classification8, 9. The sonographer or a member of the local team collated and entered the information online contemporaneously on a dedicated ultrasound reporting section of the web‐based trial management system7. The coordinating center provided on‐site training and written instructions on how to enter the data and conducted regular on‐site visits to monitor and audit data entry on the trial management system to ensure accuracy of entries. Following entry of scan findings on the trial management system, there was central automated classification of scan results based on visualization, morphology and size of simple cysts and recommended management6. Results letters were sent to the volunteers directly from the coordinating center, as were appointment letters for their next scan. There was daily regional monitoring of the automated management decisions and weekly central monitoring of any data discrepancies, with editing of incorrect entries and update of management decisions if required. All errors in data entry were fed back immediately and discussed with the regional center team coordinators and individual sonographers. Fail‐safe monitoring

Following entry of scan findings on the trial management system, there was central automated classification of scan results based on visualization, morphology and size of simple cysts and recommended management6. Results letters were sent to the volunteers directly from the coordinating center, as were appointment letters for their next scan. There was daily regional monitoring of the automated management decisions and weekly central monitoring of any data discrepancies, with editing of incorrect entries and update of management decisions if required. All errors in data entry were fed back immediately and discussed with the regional center team coordinators and individual sonographers. Fail‐safe monitoring Several monitoring mechanisms were implemented. These were overseen by the clinical fellow or the NLS, who ran a series of data queries and read all free text entered on the ultrasound report to detect: (1) discrepancies between the notes and the mandatory classification fields; (2) missing ovarian dimensions when ovaries were visualized; (3) missing descriptions of complex masses; (4) apparent errors in measurement; and (5) incidental findings. All queries were investigated by contacting the regional center team, who reviewed the written scan report and the images as appropriate. The NLS or fellow then corrected data entries on the trial management system. The regional center nurses were also alerted to ensure that all women with significant incidental findings had been referred appropriately to their general practitioner or the hospital team as per locally agreed guidelines.

es as appropriate. The NLS or fellow then corrected data entries on the trial management system. The regional center nurses were also alerted to ensure that all women with significant incidental findings had been referred appropriately to their general practitioner or the hospital team as per locally agreed guidelines. Archiving and review of images Gray‐scale images of ultrasound scans performed during the trial were archived. The protocol defined the views to be stored. For a normal scan, transverse and sagittal views of each normal ovary with measurements and a section through the uterus, including the endometrial thickness measurement, were stored. For an abnormal scan, images with measurements of any mass in two perpendicular planes were stored. If Doppler was used, the frames demonstrating the waveforms and calculations and the images of any incidental abnormalities were stored. Images were archived initially on the ultrasound machines at the centers and were then transferred weekly or biweekly (depending on volumes) on a magneto‐optical disc to the coordinating center for central archiving. Regular reviews of images of interval cancers were conducted, with feedback given to individual sonographers and discussion at the annual ultrasound meetings. In addition, the NLS or senior members of the ultrasound subcommittee reviewed all images for which concern was raised on fail‐safe monitoring of the scan reports.

gular reviews of images of interval cancers were conducted, with feedback given to individual sonographers and discussion at the annual ultrasound meetings. In addition, the NLS or senior members of the ultrasound subcommittee reviewed all images for which concern was raised on fail‐safe monitoring of the scan reports. Quality assurance monitoring Since visualization of the ovary is essential to identify any morphological abnormality of the ovary, it was considered the most important metric for QA in the trial. The sonographers were required to indicate whether each ovary was seen; was not seen but a good view of the pelvis was obtained; was not seen with a poor view of the pelvis; or was not seen owing to previous oophorectomy. It was decided that ‘visualization of the ovary’ would be the primary QA measure, as this could be verified by review of archived images when required. As we did not observe a substantial difference in VR between the right and left ovary on TVS, monitoring was based on the VR of the right ovary10. Unadjusted VRs of the right ovary were calculated on a 6‐month basis for individual sonographers undertaking more than 1000 scans during the trial period and for individual regional centers. We set a standard for observed unadjusted VR of the right ovary of 60%, based on data derived from the first 2 years of scanning in UKCTOCS and the report of VRs of one or both ovaries (data for individual ovaries were not available) from the Kentucky screening study11. Other metrics used for QA monitoring included median volume of normal ovary and missing or incorrect data in key fields, such as ovarian measurements. Reports incorporating coded individual and center data were circulated to all involved in the trial. Center leads were informed of their regional codes so that they could discuss the report in detail with their team of sonographers. In addition, the NLS contacted individual outliers for targeted training and worked with them to improve their VR.

individual and center data were circulated to all involved in the trial. Center leads were informed of their regional codes so that they could discuss the report in detail with their team of sonographers. In addition, the NLS contacted individual outliers for targeted training and worked with them to improve their VR. Statistical analysis Descriptive statistics were used to describe baseline characteristics of the women according to which sonographer performed the scan and to the center at which they were examined.

individual and center data were circulated to all involved in the trial. Center leads were informed of their regional codes so that they could discuss the report in detail with their team of sonographers. In addition, the NLS contacted individual outliers for targeted training and worked with them to improve their VR. Statistical analysis Descriptive statistics were used to describe baseline characteristics of the women according to which sonographer performed the scan and to the center at which they were examined. The VR of the right ovary was adjusted for the strongly significant (P < 0.01) non‐subjective factors identified in a previous study (age, hysterectomy, oophorectomy with intact uterus, age at menopause, tubal ligation, body mass index)10. For individual women, the ability to visualize the ovary is likely to be correlated across annual scans. The VR was therefore modeled, taking into account clustering of outcomes. To provide population‐averaged rather than individual‐level effects of covariates on visualization, we fitted a generalized estimating equation model for binary outcomes, with a logit link function and an ‘exchangeable’ correlation structure. Such a model is similar to a standard logistic regression model but with a specified intraperson correlation structure. Sonographers who had performed more than 1000 scans in total between 2001 and 2010 were included as separate fixed effects in the model. These sonographers were compared with a reference group consisting of those who had performed ≤ 1000 scans. A separate model that included the individual center effects (but with no sonographers) was also fitted. Age at first scan indicated the actual age effect, whereas scan year reflected the trend in ultrasound performance. Odds ratios for individual sonographers and centers reflected performance relative to the reference group. Comparison of model‐based adjusted VR with observed VR for individual sonographers and centers helped to indicate how the differences in the non‐subjective factors impacted on the VR of sonographers and centers. It was an indicator of the need for adjustment of VR in this setting. Rankings for the observed and adjusted VRs (1 = highest VR) reflected the impact of these adjustments.

idual sonographers and centers helped to indicate how the differences in the non‐subjective factors impacted on the VR of sonographers and centers. It was an indicator of the need for adjustment of VR in this setting. Rankings for the observed and adjusted VRs (1 = highest VR) reflected the impact of these adjustments. Annual trends of observed VR of the right ovary between 2001 and 2010 were plotted for individual sonographers and regional centers. The VRs calculated for each center included all scans performed at the center during that calendar year. Annual observed VR of the right ovary for individual sonographers was limited to sonographers who had undertaken > 1000 level‐1 TVS examinations overall and > 100 during that calendar year.

ers and regional centers. The VRs calculated for each center included all scans performed at the center during that calendar year. Annual observed VR of the right ovary for individual sonographers was limited to sonographers who had undertaken > 1000 level‐1 TVS examinations overall and > 100 during that calendar year. RESULTS Between 11 June 2001 (date of commencement of scanning in the trial) and 31 December 2010, across 13 regional centers, 48 230 of the 50 639 women in the ultrasound screening arm attended at least one screening examination and underwent a total of 293 732 annual scans. Of these, 23 697 were TAS examinations and were excluded from the analysis. The remaining 270 035 scans comprised 267 036 TVS examinations and 2999 in which both TVS and TAS were performed. The right ovary was seen in 196 426 (72.7%) scans, and was not visualized but there was a good view of the pelvis in 62 683 (23.2%) and a poor view of the pelvis in 6429 (2.4%); a previous unilateral oophorectomy was noted in 4493 (1.7%). Four scans were unrecorded. The VR of the left ovary was 69.7% (188 347/270 035) and the VR of one or both ovaries was 84.5% (228 145/270 035). The 270 035 annual scans between 2001 and 2010 included in the trial were performed by 294 sonographers. However, 222 666 (82.5%) were undertaken by 78 sonographers who had performed > 1000 scans at a center. The remaining scans (47 369) formed the reference group in the model comparing individual sonographers.

0 035). The 270 035 annual scans between 2001 and 2010 included in the trial were performed by 294 sonographers. However, 222 666 (82.5%) were undertaken by 78 sonographers who had performed > 1000 scans at a center. The remaining scans (47 369) formed the reference group in the model comparing individual sonographers. We reported previously that age at scan, age at menopause, being overweight, previous hysterectomy, previous sterilization and unilateral oophorectomy with intact uterus were significant predictors of VR10. These were used in the longitudinal analysis along with the scan year to reflect the overall trend in scanning performance (Table S1). The mean baseline characteristics of the women scanned by the 78 individual sonographers with > 1000 scans showed more variability than was found in the mean baseline characteristics of the women scanned in the different centers but with no evidence of any large systematic differences. For example, the mean age ranged between 59.5 and 63.3 years and hysterectomy rate ranged between 14.2% and 22.0% (Table S2).

h > 1000 scans showed more variability than was found in the mean baseline characteristics of the women scanned in the different centers but with no evidence of any large systematic differences. For example, the mean age ranged between 59.5 and 63.3 years and hysterectomy rate ranged between 14.2% and 22.0% (Table S2). The median adjusted VR for sonographers was 73% (interquartile range (IQR), 65–82%). The observed VR was outside the 95% CI of the adjusted VR for 38/78 (48.7%) sonographers included in the model. However the median difference between observed and adjusted VRs was only −0.7% (range, −7.9 to 5.9%). Adjustment of VR changed the attainment of the trial standard VR of 60% in only 4/78 (5.1%) sonographers, increasing the VR to ≥ 60% in two and decreasing the VR to < 60% in two. However, the impact of adjustment on VR ranking was greater, with a median change in rank of 3 (range, 0–18). The baseline characteristics of women scanned at the 13 centers (Table S3) were similar, except for Center G, which had the lowest hysterectomy rate (13.6% vs overall average of 18.1%; range, 13.6–22.0%), the lowest proportion of overweight or obese women, the second lowest rate of tubal ligation and the highest rate of infertility treatment. The median adjusted VR for the 13 centers was 74.7% (IQR, 67.1–79%), all centers having an adjusted VR above the minimum standard of 60%. The median difference between observed and adjusted VR for the 13 regional centers was −0.5% (range, −2.2 to 1%), with no change in ranking for any center (Table S4).

lity treatment. The median adjusted VR for the 13 centers was 74.7% (IQR, 67.1–79%), all centers having an adjusted VR above the minimum standard of 60%. The median difference between observed and adjusted VR for the 13 regional centers was −0.5% (range, −2.2 to 1%), with no change in ranking for any center (Table S4). Given the small differences between the observed and adjusted VRs in our trial, the former was used to determine trends over time. The annual observed VR of individual sonographers who had performed more than 1000 trial scans and of the 13 centers are detailed in Table 1 and Figure 1, respectively. The number of sonographers with VR < 60% (21.4% in 2002 vs 2.0% in 2010) decreased and those with VR > 80% (14.3% in 2002 vs 40.8% in 2010) increased over time, a trend that was more pronounced after 2006. This is reflected in the steady increase in the annual observed VR of ovaries at individual centers (Figure 1). The median center‐observed VR increased from 65.5% (range, 55.7–81.0%) in 2001 to 80.3% (range, 74.5–90.9%) in 2010. The median center‐observed VR in 2003, after all 13 participating centers became active, was 66.8%. In 2008 all centers had a VR of > 60%, seven of the 13 centers achieving overall VRs of > 80%. Table 1 Annual observed ovarian visualization rates (VR) of right ovary for 78 sonographers who had each performed > 1000 scans during United Kingdom Collaborative Trial of Ovarian Cancer Screening

Given the small differences between the observed and adjusted VRs in our trial, the former was used to determine trends over time. The annual observed VR of individual sonographers who had performed more than 1000 trial scans and of the 13 centers are detailed in Table 1 and Figure 1, respectively. The number of sonographers with VR < 60% (21.4% in 2002 vs 2.0% in 2010) decreased and those with VR > 80% (14.3% in 2002 vs 40.8% in 2010) increased over time, a trend that was more pronounced after 2006. This is reflected in the steady increase in the annual observed VR of ovaries at individual centers (Figure 1). The median center‐observed VR increased from 65.5% (range, 55.7–81.0%) in 2001 to 80.3% (range, 74.5–90.9%) in 2010. The median center‐observed VR in 2003, after all 13 participating centers became active, was 66.8%. In 2008 all centers had a VR of > 60%, seven of the 13 centers achieving overall VRs of > 80%. Table 1 Annual observed ovarian visualization rates (VR) of right ovary for 78 sonographers who had each performed > 1000 scans during United Kingdom Collaborative Trial of Ovarian Cancer Screening Year Parameter 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Number of sonographers 1 14 33 54 57 64 65 64 54 49 Observed VR < 50% — 3 (21.4)  6 (18.2)  9 (16.7)  8 (14.0)  9 (14.1)  5 (7.7)   1 (1.6)  — — 50–60% — — —  5 (9.3)  10 (17.5)  8 (12.5)  3 (4.6)   5 (7.8)   4 (7.4)   1 (2.0) 60–80% 1 (100) 9 (64.3) 15 (45.5) 20 (37.0) 20 (35.1) 30 (46.9) 34 (52.3) 35 (54.7) 30 (55.6) 28 (57.1) > 80% — 2 (14.3) 12 (36.4) 20 (37.0) 19 (33.3) 17 (26.6) 23 (35.4) 23 (35.9) 20 (37.0) 20 (40.8) Data are given as n or n (%).

6)  — — 50–60% — — —  5 (9.3)  10 (17.5)  8 (12.5)  3 (4.6)   5 (7.8)   4 (7.4)   1 (2.0) 60–80% 1 (100) 9 (64.3) 15 (45.5) 20 (37.0) 20 (35.1) 30 (46.9) 34 (52.3) 35 (54.7) 30 (55.6) 28 (57.1) > 80% — 2 (14.3) 12 (36.4) 20 (37.0) 19 (33.3) 17 (26.6) 23 (35.4) 23 (35.9) 20 (37.0) 20 (40.8) Data are given as n or n (%). Figure 1 Annual visualization rates (VR) of the right ovary by transvaginal ultrasound in 48 230 postmenopausal women attending one of 13 regional centers from 2001–2010. Thick lines indicate adjusted VR () and median observed VR (). Thin lines represent each of the 13 centers. UOG-14929-FIG-0001-bDISCUSSION Our report is the first detailed description of QA processes required for delivering TVS in large multicenter ovarian cancer screening trials or programs and for evaluating their impact on ovarian visualization. Our results show that the QA processes developed, together with regular monitoring, can lead to a high VR of postmenopausal ovaries and ensure continual improvement in VR over time. The VR showed a steady increase over the years, with the majority of sonographers and all centers achieving a VR of individual ovaries of > 60%. The differences in VR partly reflect the differing scanning abilities of individual sonographers. It is important to highlight that ovarian VR is not a perfect metric, as it underestimates pelvic visualization by not defining scans as ‘visualized’ when ovaries are not seen but there is a good view of the pelvic sidewall or when unilateral ovaries are not seen owing to previous oophorectomy.

individual sonographers. It is important to highlight that ovarian VR is not a perfect metric, as it underestimates pelvic visualization by not defining scans as ‘visualized’ when ovaries are not seen but there is a good view of the pelvic sidewall or when unilateral ovaries are not seen owing to previous oophorectomy. A key finding was that differences between observed and adjusted VRs were small. Adjustment had the greatest impact on individual sonographer ranking. This probably results from differences in non‐subjective factors being small between centers and being more pronounced between individual sonographers. The persistent difference in adjusted VR confirms that subjective factors, i.e. individual skill, attention to detail and experience, are major contributors, in keeping with previous reports that subjective assessment of the gray‐scale image has the highest accuracy12, 13, 14. It is likely that QA processes coupled with central monitoring and targeted training impacted on both non‐subjective and subjective factors.

attention to detail and experience, are major contributors, in keeping with previous reports that subjective assessment of the gray‐scale image has the highest accuracy12, 13, 14. It is likely that QA processes coupled with central monitoring and targeted training impacted on both non‐subjective and subjective factors. Ovarian visualization is key to identifying morphological abnormalities that may be indicative of ovarian cancer. Our VR of one or both ovaries was 84.5% in an analysis of 270 035 TVS examinations. This was higher than that reported in the PLCO trial5 which, like UKCTOCS, was a multicenter trial in which postmenopausal women underwent annual screening. Bodelon et al. 15 reported an ovarian VR (one or both ovaries visualized) of 53% in their most recent analysis of 102 787 scans involving 29 321 women. Both subjective and non‐subjective factors could have contributed to these differences. The rates probably reflect differences in delivery of ultrasound and may have contributed to differences in the stage (UKCTOCS Stages I/II: 48% vs PLCO Stages I/II: 22%) of screen‐detected cancers on prevalence screening in the two trials5, 6. Our VR was similar to that reported in the Kentucky ovarian cancer screening study (84%), in which 25 327 women underwent a total of 120 569 scans during an 18‐year period3. This study, unlike UKCTOCS, was limited to a single center with scanning performed by a small group of experienced personnel. Their cohort included a proportion of younger premenopausal women, in whom the presence of follicles makes ovaries easier to visualize. Neither study reported on the VR of individual ovaries as we have done for QA monitoring in UKCTOCS. Higher ovarian VR (right, 84.1%; left, 82.4%) was reported in a retrospective 5‐year audit of 6649 postmenopausal women who underwent TVS as part of clinical evaluation in a gynecological ultrasound department16. The higher rates most probably reflect the scanning expertise available at tertiary centers and also the higher incidence of adnexal pathology (which may make visualization easier) in a clinical, compared to an asymptomatic, cohort attending screening.

of clinical evaluation in a gynecological ultrasound department16. The higher rates most probably reflect the scanning expertise available at tertiary centers and also the higher incidence of adnexal pathology (which may make visualization easier) in a clinical, compared to an asymptomatic, cohort attending screening. In UKCTOCS, sonographers performed the annual level 1 scans. This reflects the practice in the UK NHS, in which sonographers perform the vast majority of pelvic ultrasound scans. It would be difficult to find the resources required for expert gynecologists to deliver primary screening in a national screening program or large multicenter trial involving > 50 000 scans yearly. However operator experience is crucial for accurate interpretation2 and patients with adnexal masses undergo fewer operations and a greater number of minimally invasive procedures, and experience shorter hospital stays, when scanned by experienced gynecologists than do those scanned by sonographers17. Hence, level 2 scans following the detection of an abnormality were performed by a senior sonographer or expert gynecologist or radiologist.

ns and a greater number of minimally invasive procedures, and experience shorter hospital stays, when scanned by experienced gynecologists than do those scanned by sonographers17. Hence, level 2 scans following the detection of an abnormality were performed by a senior sonographer or expert gynecologist or radiologist. Individual sonographer and center VRs increased over time despite the increasing age of the participants. This could partly be explained by the increasing experience of the sonographers, some of whom continued to participate in the trial for many years. However, the ongoing QA processes with their focus on sonographer training and support, regular monitoring with feedback and targeted training are likely to have contributed significantly to this increase. The SOP for examining the pelvis in postmenopausal women was introduced to sonographers at induction. It reaffirmed the principles of TVS scanning and emphasized a systematic approach to pelvic examination, identification of ovaries and optimization of images. The NLS as a quality‐control manager was key to the interaction with sonographers. In addition, the introduction of accreditation contributed to improving ultrasound quality. The British Society for Gynaecological Imaging has adopted the UKCTOCS scanning SOP as guidance for good practice and incorporated the accreditation scheme with modifications into the Society's continuing professional development program.

n, the introduction of accreditation contributed to improving ultrasound quality. The British Society for Gynaecological Imaging has adopted the UKCTOCS scanning SOP as guidance for good practice and incorporated the accreditation scheme with modifications into the Society's continuing professional development program. This report demonstrates the benefits of a formal QA process and outlines the procedures to be implemented for multicenter ultrasound screening. By introducing training days and familiarizing sonographers with the scanning protocol and SOPs, uniformity in scanning postmenopausal ovaries and interpreting gray‐scale ultrasound images was maintained. The leadership provided by the subcommittee and the attention to detail and sonographer support and training provided by the NLS were key to the sustained improvement in VR. The process of accreditation was of immediate use to those who wished to maintain standards in pelvic scanning. Limitations include the fact that QA processes were set up in the course of the trial and the NLS was appointed only midway through the trial. At the start of the trial the standard for VR of the right ovary was set at 60%, but the improvement over time suggests that this could have been set higher. In addition, independent review of the archived images to assess visualization of the ovaries is yet to be completed.

was appointed only midway through the trial. At the start of the trial the standard for VR of the right ovary was set at 60%, but the improvement over time suggests that this could have been set higher. In addition, independent review of the archived images to assess visualization of the ovaries is yet to be completed. In conclusion, if there are robust centrally implemented and monitored QA processes, ovarian VR as a performance indicator of the quality of ultrasound screening will improve with time, despite adverse factors such as aging. While VR should be adjusted for non‐subjective factors that impact on ovarian visualization, it is likely that subjective factors will be the largest contributors to VR differences. DISCLOSURES U.M. and I.J. have a financial interest through Abcodia Ltd in the third party exploitation of the trial biobank. During part of the trial, I.J. had a consultancy arrangement with Becton Dickinson in the field of tumor markers. The UKCTOCS trial was core funded by the Medical Research Council, Cancer Research UK, and the Department of Health, with additional support from the Eve Appeal, Special Trustees of Bart's and the London, and Special Trustees of University College London Hospital. The researchers at University College London were supported by the National Institute for Health Research University College London Hospitals Biomedical Research Center. The researchers are independent from the funders. Supporting information

DISCLOSURES U.M. and I.J. have a financial interest through Abcodia Ltd in the third party exploitation of the trial biobank. During part of the trial, I.J. had a consultancy arrangement with Becton Dickinson in the field of tumor markers. The UKCTOCS trial was core funded by the Medical Research Council, Cancer Research UK, and the Department of Health, with additional support from the Eve Appeal, Special Trustees of Bart's and the London, and Special Trustees of University College London Hospital. The researchers at University College London were supported by the National Institute for Health Research University College London Hospitals Biomedical Research Center. The researchers are independent from the funders. Supporting information Table S1 Generalized estimating equation model for binary outcomes for comparison of observed and adjusted visualization rates (VR) of the right ovary on ultrasound for 78 individual sonographers who had performed >  1000 scans during the United Kingdom Collaborative Trial of Ovarian Cancer Screening, including non‐subjective factors identified previously as impacting on VR Click here for additional data file. Table S2 Mean baseline characteristics of 48 230 postmenopausal women scanned by 78 individual sonographers with >  1000 scans performed during the United Kingdom Collaborative Trial of Ovarian Cancer Screening Click here for additional data file. Table S3 Mean baseline characteristics of 48 230 postmenopausal women scanned at one of 13 regional centers during the United Kingdom Collaborative Trial of Ovarian Cancer Screening

Table S2 Mean baseline characteristics of 48 230 postmenopausal women scanned by 78 individual sonographers with >  1000 scans performed during the United Kingdom Collaborative Trial of Ovarian Cancer Screening Click here for additional data file. Table S3 Mean baseline characteristics of 48 230 postmenopausal women scanned at one of 13 regional centers during the United Kingdom Collaborative Trial of Ovarian Cancer Screening Click here for additional data file. Table S4 Generalized estimating equation model for binary outcomes for comparison of observed and adjusted visualization rates (VR) of the right ovary on ultrasound at 13 regional centers participating in United Kingdom Collaborative Trial of Ovarian Cancer Screening, including non‐subjective factors identified previously as impacting on VR Click here for additional data file.

Table S4 Generalized estimating equation model for binary outcomes for comparison of observed and adjusted visualization rates (VR) of the right ovary on ultrasound at 13 regional centers participating in United Kingdom Collaborative Trial of Ovarian Cancer Screening, including non‐subjective factors identified previously as impacting on VR Click here for additional data file. ACKNOWLEDGMENTS We are very grateful to the many volunteers throughout the UK who participated in the trial and the entire medical, nursing, administrative staff and sonographers who work on the UKCTOCS. In particular, the UKCTOCS center leads: Keith Godfrey, Northern Gynaecological Oncology Center, Queen Elizabeth Hospital, Gateshead; David Oram, Department of Gynaecological Oncology, St. Bartholomew's Hospital, London; Jonathan Herod, Department of Gynaecology, Liverpool Women's Hospital, Liverpool; Karin Williamson, Department of Gynaecological Oncology, Nottingham City Hospital Nottingham; Howard Jenkins, Department of Gynaecological Oncology, Royal Derby Hospital, Derby; Tim Mould, Department of Gynaecology, Royal Free Hospital; Robert Woolas, Department of Gynaecological Oncology, St. Mary's Hospital, Portsmouth; John Murdoch, Department of Gynaecological Oncology, St. Michael's Hospital, Bristol; Stephen Dobbs, Department of Gynaecological Oncology, Belfast City Hospital, Belfast; Simon Leeson, Department of Gynaecological Oncology, Llandudno Hospital, North Wales; Derek Cruickshank, Department of Gynaecological Oncology, James Cook, University Hospital, Middlesbrough.

INTRODUCTION Hypertensive disorders, including pre‐eclampsia, occur in approximately 10% of pregnancies in the UK1. Pre‐eclampsia, a condition characterized by hypertension and proteinuria, is reported in 3% of pregnancies, and is associated with substantial perinatal morbidity and mortality in mothers and infants1, 2. The management of pre‐eclampsia is also associated with significant healthcare costs3, 4. The UK National Institute for Health and Care Excellence (NICE) guidelines recommend hospitalization for women diagnosed with pre‐eclampsia, but not for those with mild/moderate gestational hypertension only1. However, uncertainty in confirming the diagnosis leads to unnecessary admission of women with suspected but not proven pre‐eclampsia, leading to substantial healthcare costs. In a UK study between 2006 and 2008, ‘substandard care’ was linked with 20/22 deaths related to pre‐eclampsia, of which 63% were described as ‘undoubtedly avoidable’5. Moreover, timely referral to a perinatal care center was reported to reduce perinatal morbidity and mortality by 20%6. Improved diagnostic testing could reduce costs and optimize management by triaging patients at low risk of pre‐eclampsia to an outpatient setting, while ensuring that patients at moderate/high risk are managed more intensively and receive interventions (e.g. antenatal corticosteroids for fetal lung maturation) to mitigate morbidity.

ostic testing could reduce costs and optimize management by triaging patients at low risk of pre‐eclampsia to an outpatient setting, while ensuring that patients at moderate/high risk are managed more intensively and receive interventions (e.g. antenatal corticosteroids for fetal lung maturation) to mitigate morbidity. Quantification of the ratio between two angiogenic placental factors involved in the formation of new blood vessels – serum fms‐like tyrosine kinase‐1 (sFlt‐1) and placental growth factor (PlGF) – has provided valuable diagnostic information and forms the basis of the first automated biomarker test for pre‐eclampsia, the Elecsys® sFlt‐1/PlGF immunoassay ratio (Roche Diagnostics GmbH, Mannheim, Germany)7, 8, 9. PreOS, a multicenter, non‐interventional study evaluating the test for its aid in diagnosis and clinical decision‐making, found that changed decisions due to test results regarding hospitalization were in agreement with the maternal and neonatal outcomes10, 11, 12. Recently, PROGNOSIS, a global, multicenter, non‐interventional study derived and validated cut‐off values for the short‐term prediction of pre‐eclampsia. The study found that, in women presenting with clinical suspicion of pre‐eclampsia, a sFlt‐1/PlGF ratio of < 38 accurately ruled out the onset of pre‐eclampsia within 1 week13, 14, 15.

S, a global, multicenter, non‐interventional study derived and validated cut‐off values for the short‐term prediction of pre‐eclampsia. The study found that, in women presenting with clinical suspicion of pre‐eclampsia, a sFlt‐1/PlGF ratio of < 38 accurately ruled out the onset of pre‐eclampsia within 1 week13, 14, 15. The objective of this study was to estimate the incremental value of the test information measured by a reduction in expected costs of patient management because of improved accuracy in the short‐term prediction of pre‐eclampsia when using the sFlt‐1/PlGF ratio test in addition to current practice. METHODS PROGNOSIS study PROGNOSIS was a prospective, non‐interventional study conducted across 30 sites globally, including the UK, in which serum sFlt‐1/PlGF ratios were measured in 1050 women with suspected pre‐eclampsia between 24 + 0 and 36 + 6 weeks' gestation. The aim of the study was to derive and validate a cut‐off of the ratio for the short‐term prediction of pre‐eclampsia. The serum ratio levels were measured after enrollment of the derivation cohort, and at the end of the study for the validation cohort and, as such, were not available to investigators, and patient management decisions were made in the absence of the test information. Data on fetal and maternal adverse events were collected. Resource use, including planned and unplanned hospital admissions and inpatient length of stay, were also recorded13.

ion cohort and, as such, were not available to investigators, and patient management decisions were made in the absence of the test information. Data on fetal and maternal adverse events were collected. Resource use, including planned and unplanned hospital admissions and inpatient length of stay, were also recorded13. The PROGNOSIS study data were used as a source of information for the proportion of women hospitalized on suspicion of pre‐eclampsia, in the absence of test information, in current practice (‘no‐test’ scenario), the correlation between hospitalization and the test ratio and the relationship between test ratio, hospitalization and a confirmed diagnosis of pre‐eclampsia. The study was also used to provide information on inpatient length of stay, before and after the onset of pre‐eclampsia, and for women who did not develop pre‐eclampsia. In the PROGNOSIS study, each participating study site provided ethics committee/institutional review board approval of the study protocol and associated documents (participant informed consent, participant information) before the start of the clinical part of the study. All women provided written informed consent before enrollment13, 15. Being a health economic study, ethical approval was not required for the present study.

approval of the study protocol and associated documents (participant informed consent, participant information) before the start of the clinical part of the study. All women provided written informed consent before enrollment13, 15. Being a health economic study, ethical approval was not required for the present study. Model structure An economic model was developed from a UK National Health Service (NHS) payer's perspective to estimate costs associated with the diagnosis and management of a cohort of women from first presentation with clinical suspicion of pre‐eclampsia to the point of delivery. The model simulates the progression of a woman through a treatment pathway that is determined by the assessed risk of her developing pre‐eclampsia and the consequent decision to hospitalize her or to manage the pregnancy in an outpatient setting.

tation with clinical suspicion of pre‐eclampsia to the point of delivery. The model simulates the progression of a woman through a treatment pathway that is determined by the assessed risk of her developing pre‐eclampsia and the consequent decision to hospitalize her or to manage the pregnancy in an outpatient setting. The incremental value of the information generated by the test was evaluated by comparing expected management costs in two scenarios: a ‘test’ scenario (current diagnostic procedures plus the sFlt‐1/PlGF ratio) and a ‘no‐test’ scenario (current diagnostic procedures only) in a population of pregnant women presenting with a clinical suspicion of pre‐eclampsia, but in the absence of a definitive diagnosis. The incidence of pre‐eclampsia was assumed to be unaffected by the introduction of the test. Potential cost savings were expected to be driven by changes in clinical‐management strategy brought about by the test information. In particular, it was thought that the ability of the test to rule out the onset of pre‐eclampsia within 1 week may have reduced the number of women who were hospitalized unnecessarily. Treatment pathways for the test and no‐test scenarios are shown in Figure 1. Figure 1 Decision tree: (a) in the ‘no‐test’ scenario and (b) in the ‘test’ scenario.

The incremental value of the information generated by the test was evaluated by comparing expected management costs in two scenarios: a ‘test’ scenario (current diagnostic procedures plus the sFlt‐1/PlGF ratio) and a ‘no‐test’ scenario (current diagnostic procedures only) in a population of pregnant women presenting with a clinical suspicion of pre‐eclampsia, but in the absence of a definitive diagnosis. The incidence of pre‐eclampsia was assumed to be unaffected by the introduction of the test. Potential cost savings were expected to be driven by changes in clinical‐management strategy brought about by the test information. In particular, it was thought that the ability of the test to rule out the onset of pre‐eclampsia within 1 week may have reduced the number of women who were hospitalized unnecessarily. Treatment pathways for the test and no‐test scenarios are shown in Figure 1. Figure 1 Decision tree: (a) in the ‘no‐test’ scenario and (b) in the ‘test’ scenario. UOG-15997-FIG-0001-bPatient‐level data from the PROGNOSIS study provided information for each woman on whether she was hospitalized before developing pre‐eclampsia, the test ratio at baseline and whether she ultimately developed pre‐eclampsia. In the economic model, patients not admitted to hospital were assumed to be managed in an outpatient setting. Outpatient resource use was modeled by distinguishing low‐intensity management (characterized by weekly midwife‐led outpatient appointments) and intermediate‐intensity management (twice‐weekly midwife‐led outpatient appointments with some specialist involvement). Table 1 shows the main differences in resource use between the three levels of patient management (low, intermediate and high intensity).

haracterized by weekly midwife‐led outpatient appointments) and intermediate‐intensity management (twice‐weekly midwife‐led outpatient appointments with some specialist involvement). Table 1 shows the main differences in resource use between the three levels of patient management (low, intermediate and high intensity). Table 1 Modeled options for management of women with suspected pre‐eclampsia16

haracterized by weekly midwife‐led outpatient appointments) and intermediate‐intensity management (twice‐weekly midwife‐led outpatient appointments with some specialist involvement). Table 1 shows the main differences in resource use between the three levels of patient management (low, intermediate and high intensity). Table 1 Modeled options for management of women with suspected pre‐eclampsia16 Non‐hospitalized Hospitalized Low‐intensity management Intermediate‐intensity management High‐intensity management Midwife‐led hospital outpatient setting Midwife‐led hospital outpatient setting Inpatient management Average weekly appointment Average twice‐weekly appointment and specialist medical input Not applicable At each visit routine tests should be performed, including: At each visit routine tests should be performed, including: Tests including: • Blood pressure • Blood pressure • Blood pressure (four times daily) • Proteinuria • Proteinuria • Proteinuria (daily) • Blood tests • Blood tests (daily) • Kidney function • Kidney function (twice daily) • Electrolytes • Electrolytes (twice daily) • Transamines • Transamines (twice daily) • Bilirubin • Bilirubin (twice daily) No intervention Oral antihypertensive therapy twice daily Oral antihypertensive therapy twice daily Women were classified into one of three groups according to the sFlt‐1/PlGF ratio test results: < 38; 38–85; or > 85. The risk of pre‐eclampsia and probability of hospitalization were expected to be positively correlated with the value of the ratio8, 9. The lower cut‐off value of 38 to rule out pre‐eclampsia within 1 week, with a negative predictive value (NPV) of 99.3%, was derived from the PROGNOSIS study15. The higher cut‐off value of 85 for the diagnosis of pre‐eclampsia was derived from a multicenter case–control study9; according to a 2015 consensus statement, a ratio > 85 indicates that pre‐eclampsia was highly likely and administration of antenatal corticosteroids should be considered16. In the PROGNOSIS study, in which clinicians were blinded to the test ratio, 36% of women presenting with a clinical suspicion of pre‐eclampsia were hospitalized. In the absence of information on whether outpatient management was at low or intermediate intensity, the economic model assumed an equal split (32% low and 32% intermediate). Analysis of the test information from the PROGNOSIS study showed that, of the 13.2% of women with a ratio > 85, 65% were hospitalized.

pitalized. In the absence of information on whether outpatient management was at low or intermediate intensity, the economic model assumed an equal split (32% low and 32% intermediate). Analysis of the test information from the PROGNOSIS study showed that, of the 13.2% of women with a ratio > 85, 65% were hospitalized. Of the 10.7% of women with a ratio in the range 38–85, 55% were hospitalized and of the 76.1% of women with a ratio of < 38, 28% were hospitalized (Table S1).

pitalized. In the absence of information on whether outpatient management was at low or intermediate intensity, the economic model assumed an equal split (32% low and 32% intermediate). Analysis of the test information from the PROGNOSIS study showed that, of the 13.2% of women with a ratio > 85, 65% were hospitalized. Of the 10.7% of women with a ratio in the range 38–85, 55% were hospitalized and of the 76.1% of women with a ratio of < 38, 28% were hospitalized (Table S1). There is no direct information from the PROGNOSIS study on the management decisions that would have been made had the value of the ratio been known. For the purposes of modeling the test scenario, a clinical algorithm was developed to estimate the disposition of women according to the value of the ratio (Table S1), on the basis of a consensus statement on the management of pre‐eclampsia and current NICE guidelines1, 16. The conservative assumption was that, for a test ratio of > 38, the proportion of women hospitalized would be the same as was observed in the PROGNOSIS study (65% for a ratio > 85, and 55% for a ratio between 38 and 85). A ratio of < 38 denotes a low risk of pre‐eclampsia and, in principle, no woman in this group would need to be hospitalized to manage the risk. In practice, there may be other reasons for hospitalization, and the economic model is based on the assumption that a woman will be hospitalized with an sFlt‐1/PlGF ratio of < 38 and blood pressure higher than 160/110 mmHg, as recommended by current NICE guidelines. In the PROGNOSIS study, 1.7% of women met these joint criteria. All women with a ratio of > 38 were assumed to have received corticosteroids, irrespective of hospitalization, to form a conservative estimate in the economic model. The benefits of corticosteroid administration were not accounted for in the model.

elines. In the PROGNOSIS study, 1.7% of women met these joint criteria. All women with a ratio of > 38 were assumed to have received corticosteroids, irrespective of hospitalization, to form a conservative estimate in the economic model. The benefits of corticosteroid administration were not accounted for in the model. The economic model includes an option for a retest 2 weeks after the initial test, if the initial test was negative (i.e. sFlt‐1/PlGF ratio < 38). Given that the NPV of the test was still very high after 2 weeks, with a value of 97.9% (95% CI, 96.0–99.0%)14, this period was chosen in the model for the retest, despite the rule‐out period for pre‐eclampsia being 1 week in the PROGNOSIS study (with an NPV of 99.3%)16. Additional criteria for a retest were continuing symptoms of pre‐eclampsia including epigastric pain, severe edema and headache; confirmed hypertension or proteinuria; one of the criteria for HELLP syndrome; intrauterine growth restriction; or abnormal uterine perfusion.

in the PROGNOSIS study (with an NPV of 99.3%)16. Additional criteria for a retest were continuing symptoms of pre‐eclampsia including epigastric pain, severe edema and headache; confirmed hypertension or proteinuria; one of the criteria for HELLP syndrome; intrauterine growth restriction; or abnormal uterine perfusion. Costs The analysis includes the cost of the ratio test (£65), treatment costs associated with hospitalization, outpatient appointments, antihypertensive medication, regular testing, the cost of preventing complications and the cost of treating complications. The level of resource use for each of the management intensities was informed by the NICE guidelines for the management of women with hypertension in pregnancy, and unit costs were taken from UK‐specific sources (Table S2). Hospitalization costs per episode were derived from a unit cost of £2639 for a 7‐day hospital stay (£377 per day)17, multiplied by the length of stay obtained from the PROGNOSIS study for each of the treatment arms in the model. The costs of treating complications include the cost of unplanned re‐attendance of women at hospital and the cost of admission of neonates to the neonatal intensive care unit (NICU). In the absence of evidence, the model assumes that information from the test had no effect on unplanned readmissions of women, or on admission of a neonate to the NICU.

ions include the cost of unplanned re‐attendance of women at hospital and the cost of admission of neonates to the neonatal intensive care unit (NICU). In the absence of evidence, the model assumes that information from the test had no effect on unplanned readmissions of women, or on admission of a neonate to the NICU. Scenario analysis Three sets of scenario analysis were performed to test the robustness of the results: Variations in inpatient length of stay. Two separate sensitivities were run: (a) the value of all length‐of‐stay parameters was reduced by 50%; and (b) length of stay was reduced to 1.6 days for women who were hospitalized but did not develop pre‐eclampsia (in both the test and the no‐test scenarios), in line with hospital statistics for women with gestational hypertension. Variations in the proportion of women admitted to hospital, depending on the value of the test ratio: (a) the proportion of women admitted was increased by 10% and 20% for women with a ratio value of < 38 and ≥ 38 respectively; and (b) the proportion of women admitted was increased by 5% and 10% for women with a ratio of < 38. No retest. Women were tested only once, at the time of the initial suspicion of pre‐eclampsia.

Variations in the proportion of women admitted to hospital, depending on the value of the test ratio: (a) the proportion of women admitted was increased by 10% and 20% for women with a ratio value of < 38 and ≥ 38 respectively; and (b) the proportion of women admitted was increased by 5% and 10% for women with a ratio of < 38. No retest. Women were tested only once, at the time of the initial suspicion of pre‐eclampsia. RESULTS The additional information provided by the test may result in management decisions for women with suspected pre‐eclampsia that are better correlated with pre‐eclampsia outcomes than are current diagnostic procedures alone. Without the test information, 36% of women were hospitalized before a diagnosis of pre‐eclampsia, of whom 27% went on to develop pre‐eclampsia. If the additional information from the test had been available, the proportion of women hospitalized could have been reduced to around 16%, of whom 38% would have subsequently developed pre‐eclampsia. Among women who were not hospitalized, approximately the same proportion subsequently developed pre‐eclampsia. The introduction of the test is also expected to reduce the number of women hospitalized at first presentation, before developing pre‐eclampsia, from 36% to 16%. In the PROGNOSIS study population (n = 1050), this would equate to 213 fewer women hospitalized to manage the risk of pre‐eclampsia. This reduction in hospitalization would be expected to generate a cost saving of £344 per patient (8.3%) (Table 2). The additional costs of the test and retest are more than offset by savings in the cost of hospitalization. The expected annual cost savings for the UK NHS would be in the region of £24 million, based on a cohort of 68 900 women presenting annually with hypertensive disorders including suspected pre‐eclampsia18.

le 2). The additional costs of the test and retest are more than offset by savings in the cost of hospitalization. The expected annual cost savings for the UK NHS would be in the region of £24 million, based on a cohort of 68 900 women presenting annually with hypertensive disorders including suspected pre‐eclampsia18. Table 2 Cost analysis for introduction of serum fms‐like tyrosine kinase‐1/placental growth factor (sFlt‐1/PlGF) ratio test in addition to current diagnostic procedures (test scenario) compared with costs of current diagnostic procedures only (no‐test scenario), for guiding management of pre‐eclampsia (PE) in a cohort of 1050 women with suspected PE from the PROGNOSIS study Treatment No‐test scenario cost (£) Test scenario cost (£) Difference (£) Initial appointment 445 673 445 673 0 sFlt‐1/PlGF test — 68 250 68 250 sFlt‐1/PlGF retest — 40 043 40 043 Management costs prior to PE for patients who develop PE 399 103 422 755 23 652 Low risk 25 629 25 506 −123 Intermediate risk 77 169 126 907 49 738 High risk 296 306 270 343 −25 963 PE management 616 337 609 049 −7288 Management costs for patients without PE 2 811 942 2 326 603 −485 340 Low risk 304 432 351 135 46 703 Intermediate risk 916 656 1 273 271 356 616 High risk 1 590 855 702 196 −888 658 Use of corticosteroids 2737 2237 −500 Unplanned re‐attendance at hospital 69 591 69 591 — Total per cohort 4 345 382 3 984 200 −361 182 Total per patient 4138 3794 −344 Slight discrepancies between numbers and totals are due to rounding.

3 Intermediate risk 916 656 1 273 271 356 616 High risk 1 590 855 702 196 −888 658 Use of corticosteroids 2737 2237 −500 Unplanned re‐attendance at hospital 69 591 69 591 — Total per cohort 4 345 382 3 984 200 −361 182 Total per patient 4138 3794 −344 Slight discrepancies between numbers and totals are due to rounding. The rate of hospitalization derived from data for UK subjects in the PROGNOSIS study showed that the hospitalization rate in the no‐test scenario was 58%, compared with 36% in the overall PROGNOSIS cohort, indicating that there may be further potential to reduce hospitalization in the UK. Of the 44 patients hospitalized in the UK cohort, nine (20.5%) developed pre‐eclampsia compared with 27% in the overall study cohort. This may indicate that the UK is more risk‐averse with regard to hospitalization than are other countries. In the test scenario, all women with a sFlt‐1/PlGF ratio of > 38 (considered to be at intermediate or high risk of developing pre‐eclampsia) and an increased likelihood of clinical surveillance or hospitalization could be considered for antenatal corticosteroid administration in order to improve fetal lung maturation and neonatal outcome. The model conservatively accounts for the cost of corticosteroids, without quantifying the associated benefit. As such, in addition to the benefits that may be achieved by reducing unnecessary hospitalization, directed use of corticosteroids may also reduce the risk of neonatal morbidity.

turation and neonatal outcome. The model conservatively accounts for the cost of corticosteroids, without quantifying the associated benefit. As such, in addition to the benefits that may be achieved by reducing unnecessary hospitalization, directed use of corticosteroids may also reduce the risk of neonatal morbidity. Scenario analysis The overall expectation of the positive value of the sFlt‐1/PlGF ratio test in terms of reducing costs is robust to plausible changes in the main parameters. The principal effect of the information derived from the test is to reduce hospitalization, and this is the driver of cost savings. Reducing mean length of stay has the effect of reducing the value of the test from £344 to between £265 and £281 (Table 3). Similarly, increasing the proportion of women admitted to hospital also has the effect of reducing expected cost savings. With the exception of the scenario in which admission rates are increased by 10% for women with a sFlt‐1/PlGF ratio of < 38, all the scenarios remain cost saving. This is the key assumption in the analysis. Removing the retest option increases the expected cost saving from £344 to £382. Table 3 Results of scenario analyses in which serum fms‐like tyrosine kinase‐1/placental growth factor (sFlt‐1/PlGF) ratio test was used in addition to current diagnostic procedures (test scenario) and in which current diagnostic procedures only were used (no‐test scenario) for guiding management of pre‐eclampsia in a cohort of 1050 women with suspected pre‐eclampsia from the PROGNOSIS study

se‐1/placental growth factor (sFlt‐1/PlGF) ratio test was used in addition to current diagnostic procedures (test scenario) and in which current diagnostic procedures only were used (no‐test scenario) for guiding management of pre‐eclampsia in a cohort of 1050 women with suspected pre‐eclampsia from the PROGNOSIS study Cost (£) No‐test scenario Test scenario Cost difference (£) Cost difference per patient (£) Variation in LOS Base–case 4 345 382 3 984 200 −361 182 −344 LOS scenario A (halved) 4 024 584 3 729 431 −295 153 −281 LOS scenario B (1.6 days) 3 865 839 3 587 989 −277 849 −265 Percentage admitted to hospital with: Variation in number of admissions Positive test with sFlt‐1/PlGF ratio > 85 Positive test with sFlt‐1/PlGF ratio of 38–85 Negative test with sFlt‐1/PlGF ratio < 38 Cost difference per patient (£) Base–case 64.75 55.36 1.71 −344 Increase admissions by 10% (proportionately) 71.23 60.90 1.88 −290 Increase admissions by 20% (proportionately) 77.70 66.43 2.05 −235 Increase admissions of patients with a ratio < 38 by 5 percentage points 64.75 55.36 6.71 −139 Increase admissions of patients with a ratio < 38 by 10 percentage points 64.75 55.36 11.71 56 Cost (£) Variation in option of retest No‐test scenario per patient Test scenario per patient Cost difference per patient (£) Base–case £4138 £3794 −344 Exclude option of retest £4138 £3756 −382 LOS, length of stay.

Base–case 64.75 55.36 1.71 −344 Increase admissions by 10% (proportionately) 71.23 60.90 1.88 −290 Increase admissions by 20% (proportionately) 77.70 66.43 2.05 −235 Increase admissions of patients with a ratio < 38 by 5 percentage points 64.75 55.36 6.71 −139 Increase admissions of patients with a ratio < 38 by 10 percentage points 64.75 55.36 11.71 56 Cost (£) Variation in option of retest No‐test scenario per patient Test scenario per patient Cost difference per patient (£) Base–case £4138 £3794 −344 Exclude option of retest £4138 £3756 −382 LOS, length of stay. DISCUSSION Main findings Measurement of the sFlt‐1/PlGF ratio provides new information that is likely to be valuable for the short‐term prediction of pre‐eclampsia. A ratio of < 38 has a high NPV (99.3%) in ruling out the onset of pre‐eclampsia within 1 week, and this would be expected to lead to a reduction in unnecessary hospitalization in women with a clinical suspicion of pre‐eclampsia, but with no definitive diagnosis15. Our analysis of the PROGNOSIS patient‐level data shows that more than one‐third (36%) of women presenting for assessment with suspected pre‐eclampsia were admitted to hospital; however, the majority of these women did not subsequently develop pre‐eclampsia. The economic analysis quantified the impact of implementing a step‐down care approach for suspected pre‐eclampsia, taking into account the high NPV for pre‐eclampsia developing within 1 week of the sFlt‐1/PlGF ratio test.

tted to hospital; however, the majority of these women did not subsequently develop pre‐eclampsia. The economic analysis quantified the impact of implementing a step‐down care approach for suspected pre‐eclampsia, taking into account the high NPV for pre‐eclampsia developing within 1 week of the sFlt‐1/PlGF ratio test. The economic analysis suggests that introduction of the test could reduce the number of women hospitalized by more than half (56%), from 36% to 16%. The exact size of the reduction in hospitalization would depend on a number of local factors, but the general conclusion is robust to changes in all the main parameters of the economic model. The reduction in hospitalization is associated with a net saving of £344 in the base–case analysis; the additional cost of the test is more than offset by a saving in inpatient resource use. The base–case analysis includes an option to retest women who initially tested negative (sFlt‐1/PlGF ratio < 38) 2 weeks after the initial test. In the PROGNOSIS study, the proportion of women who tested negative at baseline and had not developed pre‐eclampsia but still exhibited signs and symptoms 2 weeks after the initial test was high (59%). Of those women whose retest ratio was > 38 2 weeks after the initial test, 35.5% subsequently developed pre‐eclampsia. The retest identified around 10 women (from a study cohort of 1050) at high risk of pre‐eclampsia who went on to develop the condition. Including the retest option resulted in cost savings of £344 per woman compared with £382 without retesting.

2 weeks after the initial test, 35.5% subsequently developed pre‐eclampsia. The retest identified around 10 women (from a study cohort of 1050) at high risk of pre‐eclampsia who went on to develop the condition. Including the retest option resulted in cost savings of £344 per woman compared with £382 without retesting. Strengths and limitations of the study This analysis was based on data on management and resource use, collected from a large observational study (PROGNOSIS); as such, the findings of the analysis are likely to reflect real‐world clinical practice. Moreover, consistency with clinical consensus and NICE guidelines is a strength of the analysis. Scenario analyses to determine the robustness of base–case assumptions indicated that the results were sensitive to both length of stay and the percentage of women hospitalized with a negative test result (sFlt‐1/PlGF ratio < 38). However, the conclusions of the analysis remain robust to plausible changes in model parameters. The main limitation of the analysis is the absence of a randomized interventional study on the actual impact of the test information to rule out pre‐eclampsia within 1 week in clinical practice. Although the model can simulate the effect of the most likely outcomes under a range of plausible assumptions, further research may be required to quantify the value of the sFlt‐1/PlGF ratio test information more accurately in routine practice.

t information to rule out pre‐eclampsia within 1 week in clinical practice. Although the model can simulate the effect of the most likely outcomes under a range of plausible assumptions, further research may be required to quantify the value of the sFlt‐1/PlGF ratio test information more accurately in routine practice. A limitation from the perspective of the UK healthcare system is that PROGNOSIS includes data from countries other than the UK, whose protocols may differ from those of the UK. However, management is expected to be similar in the UK to that in other countries, and indeed, in PROGNOSIS, a higher overall hospitalization rate was observed in the UK study center. Overall, therefore, the limitations of the model are not expected to impact significantly on the findings of the study.

hose of the UK. However, management is expected to be similar in the UK to that in other countries, and indeed, in PROGNOSIS, a higher overall hospitalization rate was observed in the UK study center. Overall, therefore, the limitations of the model are not expected to impact significantly on the findings of the study. Interpretation in light of other evidence The impact of the sFlt‐1/PlGF ratio test as an aid to diagnosis and clinical decision‐making has been investigated in Austria and Germany in the PreOS study, where the test is in routine clinical use in accordance with their local guidelines10, 11, 12, 19. The cut‐off ratio of 85 was considered in PreOS to confirm the diagnosis of pre‐eclampsia and inform management of women with suspected pre‐eclampsia. The change in clinical management observed in PreOS was consistent with the assumptions made in this analysis of the PROGNOSIS data, in which it was assumed that patients with sFlt‐1/PlGF ratios indicating low risk would be managed less intensively than patients who were indicated to be at moderate/high risk. Resource savings based on the use of the test as an aid to diagnosis were reported previously in a UK‐based analysis published in 2010, which reported savings of £945 per patient using the sFlt‐1/PlGF ratio test20.

low risk would be managed less intensively than patients who were indicated to be at moderate/high risk. Resource savings based on the use of the test as an aid to diagnosis were reported previously in a UK‐based analysis published in 2010, which reported savings of £945 per patient using the sFlt‐1/PlGF ratio test20. A consensus publication from 2015 stated that, in women with a particularly high sFlt‐1/PlGF ratio, there was an association with a need to deliver the infant within 48 h, and thus close surveillance and prompt initiation of corticosteroids were strongly recommended16. These data emphasize the clinical need to identify accurately women at high risk of complications from pre‐eclampsia, as well as the economic need to reduce unnecessary hospitalization of women at low risk.

nt within 48 h, and thus close surveillance and prompt initiation of corticosteroids were strongly recommended16. These data emphasize the clinical need to identify accurately women at high risk of complications from pre‐eclampsia, as well as the economic need to reduce unnecessary hospitalization of women at low risk. NICE assessed PlGF‐based testing to help diagnose suspected pre‐eclampsia21. The economic model showed cost reductions per patient compared with standard clinical assessment of £2488 for the Elecsys immunoassay sFlt‐1/PlGF ratio for women presenting with suspected pre‐eclampsia before 35 weeks' gestation, and NICE diagnostic guidance recommends the Elecsys immunoassay sFlt‐1/PlGF ratio to help rule out pre‐eclampsia and avoid unnecessary hospital admissions21. The NICE assessment focused on women with suspected pre‐eclampsia before 35 weeks' gestation (the PROGNOSIS study and the present analysis included women with suspected pre‐eclampsia between 24 and 36 + 6 weeks' gestation) and also included different sources for clinical inputs and costs of NICU stay, resulting in greater cost reductions than those found in the current analysis. The NICE assessment confirms in a separately developed model the cost‐saving potential of the sFlt‐1/PlGF ratio test in women with suspected pre‐eclampsia. Conclusion This study demonstrates that use of the sFlt‐1/PlGF ratio test in the UK may lead to a reduction in unnecessary hospitalization for women with suspected pre‐eclampsia, resulting in substantial cost savings. Supporting information Table S1 Clinical algorithm used in the model

NICE assessed PlGF‐based testing to help diagnose suspected pre‐eclampsia21. The economic model showed cost reductions per patient compared with standard clinical assessment of £2488 for the Elecsys immunoassay sFlt‐1/PlGF ratio for women presenting with suspected pre‐eclampsia before 35 weeks' gestation, and NICE diagnostic guidance recommends the Elecsys immunoassay sFlt‐1/PlGF ratio to help rule out pre‐eclampsia and avoid unnecessary hospital admissions21. The NICE assessment focused on women with suspected pre‐eclampsia before 35 weeks' gestation (the PROGNOSIS study and the present analysis included women with suspected pre‐eclampsia between 24 and 36 + 6 weeks' gestation) and also included different sources for clinical inputs and costs of NICU stay, resulting in greater cost reductions than those found in the current analysis. The NICE assessment confirms in a separately developed model the cost‐saving potential of the sFlt‐1/PlGF ratio test in women with suspected pre‐eclampsia. Conclusion This study demonstrates that use of the sFlt‐1/PlGF ratio test in the UK may lead to a reduction in unnecessary hospitalization for women with suspected pre‐eclampsia, resulting in substantial cost savings. Supporting information Table S1 Clinical algorithm used in the model Table S2 Treatment costs to prevent complications in women with pre‐eclampsia, stratified by management intensity Click here for additional data file.

Conclusion This study demonstrates that use of the sFlt‐1/PlGF ratio test in the UK may lead to a reduction in unnecessary hospitalization for women with suspected pre‐eclampsia, resulting in substantial cost savings. Supporting information Table S1 Clinical algorithm used in the model Table S2 Treatment costs to prevent complications in women with pre‐eclampsia, stratified by management intensity Click here for additional data file. ACKNOWLEDGMENTS We thank John Posnett of PAREXEL for advice and input into the economic model and Fiona Sheppard, Michael Tang and John Posnett (all with PAREXEL) for writing support. PAREXEL provided consultancy services funded by Roche Diagnostics. Elecsys and Cobas are trademarks of Roche Diagnostics. C.W., T.S.M., M.H. and D.A. are employees of Roche Diagnostics.

INTRODUCTION Massively parallel next‐generation whole‐genome sequencing (WGS) of cell‐free DNA (cfDNA) from maternal plasma has been shown to have high sensitivity and specificity for the detection of trisomies 21, 18 and 13, and for sex chromosome analysis in singleton pregnancies1, 2, 3, 4. However, twins account for approximately 1 in 30 live births in the USA, and the rate of twin births is increasing5. With a high proportion of twin births thought to originate in women undergoing assisted reproductive technology (ART), the use of non‐invasive prenatal testing (NIPT) to screen for fetal aneuploidy is especially desirable. Traditionally, prenatal aneuploidy screening options have been less robust for twin pregnancies than for singletons6, whereas the miscarriage risk associated with invasive diagnostic procedures is higher in twins7. Preliminary data have suggested that NIPT is a feasible test option for twin gestations8, 9, 10. Currently, due to the paucity of reported studies in twins, professional societies and others have called for more studies on NIPT performance in twin gestations11, 12, 13, 14.

ive diagnostic procedures is higher in twins7. Preliminary data have suggested that NIPT is a feasible test option for twin gestations8, 9, 10. Currently, due to the paucity of reported studies in twins, professional societies and others have called for more studies on NIPT performance in twin gestations11, 12, 13, 14. One of the factors governing NIPT performance is the fetal contribution to the cfDNA present in maternal plasma, known as the fetal fraction, with NIPT being offered from around 10 weeks of gestation because of the lower fetal contribution at earlier gestational ages. Although the total fetal fraction has been shown to be as much as 35% higher in twin pregnancies when compared with singletons10, the fetal fraction per twin is lower8, 9. Furthermore, it has been shown that individual cfDNA contribution from each twin could differ by as much as two‐fold15, 16. The complexity of the fetal fraction in twin gestations has raised concerns about a potentially increased false‐negative rate of NIPT in twin gestations. Furthermore, single‐nucleotide polymorphism‐based technologies or targeted sequencing technologies for NIPT are not currently offered clinically for twin gestations. The primary objective of this study was to describe the clinical laboratory experience of a WGS‐based NIPT in twin gestations. The secondary objective was to estimate the fetal fraction in clinical samples from twin pregnancies.

One of the factors governing NIPT performance is the fetal contribution to the cfDNA present in maternal plasma, known as the fetal fraction, with NIPT being offered from around 10 weeks of gestation because of the lower fetal contribution at earlier gestational ages. Although the total fetal fraction has been shown to be as much as 35% higher in twin pregnancies when compared with singletons10, the fetal fraction per twin is lower8, 9. Furthermore, it has been shown that individual cfDNA contribution from each twin could differ by as much as two‐fold15, 16. The complexity of the fetal fraction in twin gestations has raised concerns about a potentially increased false‐negative rate of NIPT in twin gestations. Furthermore, single‐nucleotide polymorphism‐based technologies or targeted sequencing technologies for NIPT are not currently offered clinically for twin gestations. The primary objective of this study was to describe the clinical laboratory experience of a WGS‐based NIPT in twin gestations. The secondary objective was to estimate the fetal fraction in clinical samples from twin pregnancies. METHODS Patients and sample collection Clinical Study A included frozen plasma samples from twin pregnancies with known outcomes that were collected as part of two independent clinical studies, MELISSA (MatErnal BLood IS Source to Accurately diagnose fetal aneuploidy) and CARE (Comparison of Aneuploidy Risk Evaluations), in high‐risk and all‐risk pregnant populations, as described previously1, 17. Briefly, a minimum of 7 mL of whole blood was collected in acid citrate dextrose1 or cfDNA blood‐collection (Streck) tubes17 and shipped either overnight in temperature‐controlled (cooled) conditions (acid citrate dextrose tubes) or in ambient shippers within 5 days of blood draw (Streck tubes) to the Illumina R&D Laboratory (Redwood City, CA, USA), where samples were inspected and plasma was prepared and stored at –80°C until sequencing. cfDNA was isolated from the plasma by centrifugation at 1600 g for 10 min. A second 10‐min centrifugation step was performed on the supernatant after transfer to a fresh tube or plate1, 17, 18, 19. Institutional review boards at each collection site approved the studies and written informed consent was obtained from each patient. In both studies, women ≥ 18 years of age with a twin pregnancy ≥ 8 weeks' gestation were eligible for inclusion; 1.7% (n = 2) of samples were obtained at < 10 weeks' gestation. For the MELISSA study, patients with a gestational age > 22 weeks were excluded and eligibility criteria did not require an invasive procedure (amniocentesis or chorionic villus sampling) prior to enrollment1. For the CARE study, eligibility criteria required that a minimum of 2 weeks had elapsed between an invasive procedure, if performed, and the blood draw for cfDNA testing17; none of the samples in this study was drawn after an invasive procedure. Patients in the CARE study were also excluded if prenatal screening for aneuploidy was carried out by measurement of nuchal translucency only. Data on clinical outcome detailing fetal karyotype from invasive prenatal procedures and/or newborn physical examination were entered into an electronic database by research personnel.

s in the CARE study were also excluded if prenatal screening for aneuploidy was carried out by measurement of nuchal translucency only. Data on clinical outcome detailing fetal karyotype from invasive prenatal procedures and/or newborn physical examination were entered into an electronic database by research personnel. Research laboratory personnel who carried out the sequencing were blinded to the clinical outcome data. Classification by sequencing was compared with clinical outcome for all subjects; cfDNA‐based NIPT results were not reported to the patients.

s in the CARE study were also excluded if prenatal screening for aneuploidy was carried out by measurement of nuchal translucency only. Data on clinical outcome detailing fetal karyotype from invasive prenatal procedures and/or newborn physical examination were entered into an electronic database by research personnel. Research laboratory personnel who carried out the sequencing were blinded to the clinical outcome data. Classification by sequencing was compared with clinical outcome for all subjects; cfDNA‐based NIPT results were not reported to the patients. Clinical Study B included fresh maternal blood samples indicated as twin gestation on the test requisition forms (TRFs) that were received during the study period at the College of American Pathologists‐accredited and Clinical Laboratory Improvement Act‐certified Illumina Laboratory from providers in the USA requesting the commercially available verifi® Prenatal Test (Illumina, Inc., San Diego, CA, USA). Samples received from distributor laboratories and/or health systems located in the USA were excluded due to the inability to obtain clinical follow‐up. Although unlikely, it is possible that some of the samples in Clinical Study B were obtained after an invasive procedure (for which the timing is unknown). Demographic information, such as maternal age, gestational age and clinical indication for testing, was obtained from the TRF. Reasons for testing indicated on the TRF were the following: advanced maternal age (AMA), abnormal ultrasound finding, previous affected pregnancy or positive serum screen (determined by local community standards); clinicians could select multiple options. NIPT yielded a report that was sent to physicians with results of aneuploidy status for chromosomes 21, 18 and 13 (‘aneuploidy detected’, ‘no aneuploidy detected’ or ‘aneuploidy suspected’, as described previously1) and presence of Y (‘detected’ or ‘not detected’), if requested. Testing of clinical samples could be canceled due to either administrative (insufficient sample quantity, gestational age at sampling < 10 weeks or patient or physician request) or technical (failure to meet quality‐control metrics, laboratory processing issue or insufficient or high cfDNA concentration) reasons18, 20. Providers were notified if the test was canceled and offered the option to submit a second sample.

ntity, gestational age at sampling < 10 weeks or patient or physician request) or technical (failure to meet quality‐control metrics, laboratory processing issue or insufficient or high cfDNA concentration) reasons18, 20. Providers were notified if the test was canceled and offered the option to submit a second sample. Sample preparation and analysis cfDNA was extracted from frozen (Clinical Study A) and fresh (Clinical Study B) plasma samples; 1 mL of plasma was required for analysis. Sequencing libraries were prepared using the Illumina TruSeq DNA Sample Prep Kit and sequencing was performed on Illumina HiSeq 2000 sequencers (Illumina, Inc.), as described previously for singleton pregnancies1, 17, 18. Sequence alignment and tag counting methods have been described previously1, 19. Analysis was performed using SAFeR (Selective Algorithm for Fetal Results), which incorporated several important updates to the analytic platform, including improved genomic filtering, removal of systematic biases and improved normalization and classification approaches. These updates were designed to improve the assay limitation of detection, theoretically enabling sufficient sensitivity for aneuploidy detection in twin gestations.

ates to the analytic platform, including improved genomic filtering, removal of systematic biases and improved normalization and classification approaches. These updates were designed to improve the assay limitation of detection, theoretically enabling sufficient sensitivity for aneuploidy detection in twin gestations. Fetal fraction estimates were made using tags on the X chromosome and/or chromosome 21, as described previously21. Fetal fraction estimates based on X chromosome tags were made for samples in which aneuploidy was not detected and the presence of Y was detected, and for samples reported as aneuploidy detected for trisomy 21 with presence of Y for which fetal karyotype was known. Fetal fraction estimates using tags on chromosome 21 were performed for samples reported as trisomy 21 detected for which fetal karyotype or birth outcome was known. Fetal fraction estimates were not determined for samples that did not have either presence of Y or trisomy 21 detected. These estimates were made for the purpose of this study analysis only; no fetal fraction cut‐off limit for reporting was applied to samples. At the time of the study, fetal fraction estimates were not reported to patients by the Illumina laboratory.

samples that did not have either presence of Y or trisomy 21 detected. These estimates were made for the purpose of this study analysis only; no fetal fraction cut‐off limit for reporting was applied to samples. At the time of the study, fetal fraction estimates were not reported to patients by the Illumina laboratory. Clinical outcomes An active follow‐up process was utilized to obtain information on fetal karyotype and sex (confirmed by invasive diagnostic procedure, newborn testing/physical examination or ultrasound evaluation) for all clinical cases according to standard laboratory practice and quality procedures, as described previously18, 22. The estimated delivery date had passed for all samples at the time of outcome collection. Fetal sex of each twin was requested for all cases in which sex chromosome status by NIPT was ordered on the TRF.

all clinical cases according to standard laboratory practice and quality procedures, as described previously18, 22. The estimated delivery date had passed for all samples at the time of outcome collection. Fetal sex of each twin was requested for all cases in which sex chromosome status by NIPT was ordered on the TRF. When aneuploidy was detected or suspected for chromosomes 13, 18 and/or 21, cases were categorized as (1) ‘concordant’ if NIPT results matched the karyotype or birth outcome for one or both twins (true positive); (2) ‘discordant’ if NIPT results did not match the karyotype or birth outcome of either twin (false positive); (3) ‘suspected to be concordant’ if karyotype information was unavailable but other indicators suggestive of aneuploidy, such as abnormal ultrasound findings, were present; or (4) ‘no information’ if outcome information was insufficient to determine or suspect concordance, or was not available to the laboratory because a practice failed to respond to our request for outcomes. When aneuploidy was not detected, cases were categorized as (1) ‘concordant’ if both twins were determined to be unaffected by karyotyping or birth outcome (true negative); (2) ‘discordant’ if one or both twins were determined to be affected by karyotyping or birth outcome (false negative); or (3) ‘no information’, as above.

neuploidy was not detected, cases were categorized as (1) ‘concordant’ if both twins were determined to be unaffected by karyotyping or birth outcome (true negative); (2) ‘discordant’ if one or both twins were determined to be affected by karyotyping or birth outcome (false negative); or (3) ‘no information’, as above. For fetal sex, reported as the presence or absence of Y, cases were categorized as (1) ‘concordant’ if presence of Y was reported and one or both twins were determined from clinical outcomes or ultrasound evaluation to have a Y chromosome (XY, XXY or XYY), or absence of Y was reported and both twins lacked the Y chromosome (monosomy X, XX or XXX); (2) ‘discordant’ if presence of Y was reported and both twins lacked a Y chromosome (monosomy X, XX or XXX), or absence of Y was reported and one or both twins was determined to have a Y chromosome (XY, XXY or XYY); or (3) ‘no information’, as above. Sex chromosome abnormality testing was not available to patients with twin gestations; these were reported as presence or absence of Y only. Statistical analysis Statistical significance was determined by an unpaired Student's t‐test for continuous variables, a chi‐square test for categorical variables and the Wilcoxon test for non‐parametric values. A P‐value < 0.05 was considered statistically significant. Analyses were performed using the R statistical package (version 2.12.0; R Foundation for Statistical Computing, Vienna, Austria) and Microsoft Excel statistical tool.

a chi‐square test for categorical variables and the Wilcoxon test for non‐parametric values. A P‐value < 0.05 was considered statistically significant. Analyses were performed using the R statistical package (version 2.12.0; R Foundation for Statistical Computing, Vienna, Austria) and Microsoft Excel statistical tool. Results Demographic characteristics Demographic characteristics of the two study cohorts of Clinical Studies A and B are shown in Table 1. Mean maternal age was similar between the two cohorts. Mean gestational age at sampling was significantly lower in Clinical Study B (P < 0.0001). Table 1 Demographic characteristics of twin pregnancies with non‐invasive prenatal testing included in Clinical Studies A and B Characteristic Clinical Study A (n = 115) Clinical Study B (n = 487) P Maternal age (years) 34.4 ± 6.1 (18.9–48.9) 35.5 ± 4.9 (18.3–53.5)  0.1016 Gestational age (weeks) 16.6 ± 6.5 (8–35) 13.7 ± 3.9 (9–32)* < 0.0001 Trimester < 0.0001 First (≤ 13 weeks) 55 (47.8) 333 (68.4) Second (14–27 weeks) 49 (42.6) 149 (30.6) Third (28–40 weeks) 11 (9.6)   5 (1.0) Data are given as mean ± SD (range) or n (%). * Testing of samples obtained at < 10 weeks' gestation was canceled.

Maternal age (years) 34.4 ± 6.1 (18.9–48.9) 35.5 ± 4.9 (18.3–53.5)  0.1016 Gestational age (weeks) 16.6 ± 6.5 (8–35) 13.7 ± 3.9 (9–32)* < 0.0001 Trimester < 0.0001 First (≤ 13 weeks) 55 (47.8) 333 (68.4) Second (14–27 weeks) 49 (42.6) 149 (30.6) Third (28–40 weeks) 11 (9.6)   5 (1.0) Data are given as mean ± SD (range) or n (%). * Testing of samples obtained at < 10 weeks' gestation was canceled. Clinical Study A Clinical Study A included frozen plasma samples from 115 twin gestations: three from trisomy 21‐affected pregnancies, one from a trisomy 18‐affected pregnancy and 111 from unaffected twin gestations (Table 2). In this cohort, 53.0% (61/115) of twin pregnancies were patients undergoing ART, including six cases of ovum donor pregnancies. Conventional prenatal aneuploidy screening results (serum biomarkers ± ultrasound measure of nuchal translucency thickness), carried out using various screening types including sequential and first‐trimester combined testing, were available for 82 patients. Of these results, 20 (24.4%) were false positives and two (2.4%) were false negatives; of the false‐positive results, 90% (18/20) were for trisomy 21 and 10% (2/20) were for trisomy 18. Trisomy 18 results for twin gestations could not be provided by conventional screening approaches in 32.9% (27/82) of patients, depending on the laboratory and type of screening used. In comparison, all 115 samples in Clinical Study A were sequenced and analyzed, and generated an NIPT result (Table 2). All four samples from pregnancies with at least one aneuploid fetus were identified correctly by NIPT for the appropriate aneuploid chromosome (sensitivity, 100%); one of these cases was from a patient undergoing ART. These four affected samples were obtained at an average gestational age of 17.3 (range, 11.6–32.9) weeks. None of the clinically defined unaffected samples was classified as aneuploidy detected or suspected (specificity, 100%). Additionally, the presence or absence of the Y chromosome was identified correctly in all samples (Table 2).

ted samples were obtained at an average gestational age of 17.3 (range, 11.6–32.9) weeks. None of the clinically defined unaffected samples was classified as aneuploidy detected or suspected (specificity, 100%). Additionally, the presence or absence of the Y chromosome was identified correctly in all samples (Table 2). Table 2 Results of non‐invasive prenatal testing (NIPT) in 115 twin pregnancies with known clinical outcome included in Clinical Study A Sample size (n) Clinical outcome NIPT result Twin A Twin B Aneuploidy Chromosome Y 24 46,XX 46,XX Not detected Absent 45 46,XX 46,XY Not detected Present 42 46,XY 46,XY Not detected Present 2 47,XY + 21 46,XY T21 detected Present 1 Mosaic 47,XY + 21 [7]/46,XY [11] 46,XX T21 detected Present 1 47,XY + 18 47,XY + 18 T18 detected Present DR: 91/91 (100%); Spec: 24/24 (100%) DR, detection rate; Spec, specificity; T18, trisomy 18; T21, trisomy 21.

46,XY Not detected Present 42 46,XY 46,XY Not detected Present 2 47,XY + 21 46,XY T21 detected Present 1 Mosaic 47,XY + 21 [7]/46,XY [11] 46,XX T21 detected Present 1 47,XY + 18 47,XY + 18 T18 detected Present DR: 91/91 (100%); Spec: 24/24 (100%) DR, detection rate; Spec, specificity; T18, trisomy 18; T21, trisomy 21. Clinical Study B A total of 487 fresh maternal blood samples from twin gestations met the inclusion criteria for the clinical outcome study. Zygosity, chorionicity and method of conception were not reported to the laboratory for most samples. NIPT results were reported in 98.4% (479/487) of cases with an average turnaround time of 3.2 business days. Eight (1.6%) tests were canceled, none due to technical reasons. Of the 479 reported NIPT results, seven (1.5%) were reported as aneuploidy detected for trisomy 21, one (0.2%) as aneuploidy suspected for trisomy 21, one (0.2%) as aneuploidy detected for trisomy 21 and aneuploidy suspected for trisomy 18 and 470 (98.1%) as no aneuploidy detected. Fetal sex determination was requested in 87.5% (419/479) of cases. Of these, presence of Y was reported in 70.9% (297/419) and absence of Y reported in 29.1% (122/419).

ted for trisomy 21, one (0.2%) as aneuploidy detected for trisomy 21 and aneuploidy suspected for trisomy 18 and 470 (98.1%) as no aneuploidy detected. Fetal sex determination was requested in 87.5% (419/479) of cases. Of these, presence of Y was reported in 70.9% (297/419) and absence of Y reported in 29.1% (122/419). Information on aneuploidy outcome was available for 171 (35.7%) cases; no false negatives were reported. Of the nine cases reported as aneuploidy detected or suspected, six were confirmed as true positives, one was a false positive and two were unconfirmed but were suspected to be concordant based on ultrasound findings (Table 3). All aneuploidy detected cases were either confirmed or had ultrasound findings suggestive of aneuploidy; the one discordant result was an aneuploidy suspected case. In this single discordant case, a fetal microduplication of 1.27 megabases on chromosome 20q11 was reported to the laboratory; the significance of this finding with respect to the discordant NIPT result is unclear. No information on maternal genotype was available for this case so we were unable to determine whether the microduplication was maternally inherited. Both unconfirmed aneuploidy detected/suspected cases, including the double aneuploidy case, had ultrasound findings that could be consistent with the NIPT result; in one case there was demise of the twin post blood draw. Based on the one reported false‐positive case, the observed false‐positive frequency in the clinical study population was 0.2% (1/479); if the two unconfirmed cases were also false positives, the false‐positive frequency would have been 0.6% (3/479). However, for aneuploidy detected cases only, the observed false‐positive frequency was 0.0% (0/479).

‐positive case, the observed false‐positive frequency in the clinical study population was 0.2% (1/479); if the two unconfirmed cases were also false positives, the false‐positive frequency would have been 0.6% (3/479). However, for aneuploidy detected cases only, the observed false‐positive frequency was 0.0% (0/479). Table 3 Clinical outcome in nine twin pregnancies with non‐invasive prenatal testing (NIPT) result of aneuploidy detected or suspected included in Clinical Study B Sample NIPT result Clinical outcome Aneuploidy Chromosome Y Twin A Twin B Source Details 1 T21 detected Present XY + 21 XY CVS 2 T21 detected Present XY + 21 XY CVS 3 T21 detected Present XY + 21 XX Amniocentesis Triplet to dizygotic twins 4 T21 detected Present XY + 21 XX Amniocentesis 5 T21 detected Present XX + 21 XY Visual exam at birth 6 T21 detected NA* XY + 21 XX Cord blood analysis 7 T21 suspected Present XY XX Amniocentesis 1.27 Mb microduplication of 20q11 in one fetus 8 T21 detected Absent Abnormal findings† Normal findings Ultrasound exam 9 T21 detected, T18 suspected Present Demise NA Of the six clinical cases with confirmed fetal trisomy, three underwent selective reduction of affected twin, two delivered both twins and the other case began as a triplet pregnancy with demise of one fetus at 6 weeks; amniocentesis of the remaining two viable fetuses confirmed trisomy 21 in one and normal karyotype in the other. * Fetal sex information not requested. † Included increased nuchal translucency, absent stomach, pyelectasis and polyhydramnios.

T18 suspected Present Demise NA Of the six clinical cases with confirmed fetal trisomy, three underwent selective reduction of affected twin, two delivered both twins and the other case began as a triplet pregnancy with demise of one fetus at 6 weeks; amniocentesis of the remaining two viable fetuses confirmed trisomy 21 in one and normal karyotype in the other. * Fetal sex information not requested. † Included increased nuchal translucency, absent stomach, pyelectasis and polyhydramnios. CVS, chorionic villus sampling; Mb, megabases; NA, not available; T, trisomy. Of the 419 cases for which fetal sex information was requested, 156 (37.2%) had this information reported to the laboratory (Table 4); fetal sex was identified by ultrasound in 16 of these 156 cases. For three cases with a NIPT report indicating an absence of Y chromosome, one twin was confirmed to be female but the sex of the other twin was not reported because of cotwin demise. Of the 153 cases with sufficient clinical information for comparison with NIPT results, concordance for the presence of Y (one or both twins were male) was confirmed in 99.1% (115/116) of cases and concordance for absence of Y (both twins were female) was confirmed in 97.3% (36/37); there were two cases of fetal gender discordance reported (Table 4). Therefore, in the majority of cases, presence or absence of Y was indicative of the true status of the fetuses. Table 4 Results of non‐invasive prenatal testing (NIPT) for presence or absence of Y chromosome in 479 twin pregnancies included in Clinical Study B

Of the 419 cases for which fetal sex information was requested, 156 (37.2%) had this information reported to the laboratory (Table 4); fetal sex was identified by ultrasound in 16 of these 156 cases. For three cases with a NIPT report indicating an absence of Y chromosome, one twin was confirmed to be female but the sex of the other twin was not reported because of cotwin demise. Of the 153 cases with sufficient clinical information for comparison with NIPT results, concordance for the presence of Y (one or both twins were male) was confirmed in 99.1% (115/116) of cases and concordance for absence of Y (both twins were female) was confirmed in 97.3% (36/37); there were two cases of fetal gender discordance reported (Table 4). Therefore, in the majority of cases, presence or absence of Y was indicative of the true status of the fetuses. Table 4 Results of non‐invasive prenatal testing (NIPT) for presence or absence of Y chromosome in 479 twin pregnancies included in Clinical Study B Clinical sex NIPT result Total Presence of Y Absence of Y Sex not requested XX   0   3 0 3 XX/XX   1  36 2 39 XY   4   0 0 4 XX/XY  58   1 2 61 XX/XXY   1   0 0 1 XY/XY  52   0 1 53 No information 181  82 55 318 Total 297 122 60 479 DR: 115/116 (99.1%)* Spec: 36/37 (97.3%)* * Excludes cases with insufficient outcome data to determine detection rate (DR) and specificity (Spec).

Clinical sex NIPT result Total Presence of Y Absence of Y Sex not requested XX   0   3 0 3 XX/XX   1  36 2 39 XY   4   0 0 4 XX/XY  58   1 2 61 XX/XXY   1   0 0 1 XY/XY  52   0 1 53 No information 181  82 55 318 Total 297 122 60 479 DR: 115/116 (99.1%)* Spec: 36/37 (97.3%)* * Excludes cases with insufficient outcome data to determine detection rate (DR) and specificity (Spec). Fetal fraction Fetal fraction estimates were calculated for clinical samples for which the presence of Y was reported as detected and fetal gender outcome was known for both twins (Figure 1). For twin gestations with one male and one female, the fetal fraction being measured was that of the male fetus. For twin gestations with two male fetuses, the fetal fraction being measured was the combined fetal fraction from the two male fetuses. The average fetal fraction in samples from pregnancies with one female (XX) and one male (XY) twin was 7.8 ± 4.0% (range, 0.8–17.4%; n = 58). The average fetal fraction in samples from pregnancies with two confirmed male (XY) twins was 16.1 ± 6.7% (range, 2.8–31.9%; n = 51). Figure 1 Fetal fraction of cell‐free DNA in maternal plasma according to gestational age at non‐invasive prenatal testing in 58 twin pregnancies with one male and one female fetus (XX/XY, ) and 51 with two male fetuses (XY/XY, ), confirmed by clinical outcome. The XX/XY fetal fraction estimate is derived solely from the male twin, while the XY/XY fetal fraction is the combined value for both twins.

nal age at non‐invasive prenatal testing in 58 twin pregnancies with one male and one female fetus (XX/XY, ) and 51 with two male fetuses (XY/XY, ), confirmed by clinical outcome. The XX/XY fetal fraction estimate is derived solely from the male twin, while the XY/XY fetal fraction is the combined value for both twins. UOG-15964-FIG-0001-bFetal fraction estimation using chromosome 21 and the X chromosome was performed in six samples with confirmed trisomy 21 karyotype and at least one male fetus (Table 5). Fetal fraction measurement using chromosome 21 provides an estimate of the fetal fraction derived from the affected twin only. Similarly, fetal fraction estimation using the X chromosome provides an estimate of the fetal fraction derived from fetuses carrying one X chromosome (in this case, male fetuses). Table 5 Fetal fraction (FF) estimates in six twin pregnancies with confirmed trisomy 21 and at least one male fetus, included in Clinical Study B Fetal karyotype (Twin A/B) FF by X (%) FF by Chr 21 (%) Source of FF Correlation with clinical findings XY/XY + 21 31.9 12.0 FF by X is from Twins A and B FF by Chr 21 is from Twin B Expectation: higher FF by X* Finding: higher FF by X XY/XY + 21 13.5 6.4 FF by X is from Twins A and B FF by Chr 21 is from Twin B Expectation: higher FF by X* Finding: higher FF by X XX/XY + 21 11.3 4.8 Both FF measurements are from Twin B Expectation: similar FF† Finding: higher FF by X‡ XX/XY + 21 6.7 7.2 Both FF measurements are from Twin B Expectation: similar FF† Finding: similar FF XY/XX + 21 6.6 9.6 FF by X is from Twin A

FF by Chr 21 is from Twin B Expectation: higher FF by X* Finding: higher FF by X XX/XY + 21 11.3 4.8 Both FF measurements are from Twin B Expectation: similar FF† Finding: higher FF by X‡ XX/XY + 21 6.7 7.2 Both FF measurements are from Twin B Expectation: similar FF† Finding: similar FF XY/XX + 21 6.6 9.6 FF by X is from Twin A FF by Chr 21 is from Twin B Expectation: FF within 1.5 of each other§ Finding: FF within 1.5 of each other XX/XY + 21 15.1 14.8 Both FF measurements are from Twin B Expectation: similar FF† Finding: similar FF * FF by X is combined contribution from both twins and FF by chromosome (Chr) 21 is only from affected twin. † Because both FF measurements are from same twin, we would expect similar values for each method. ‡ Originally triplet pregnancy with one demise at around 6 weeks; if the demised twin was male, without trisomy 21, this could account for higher FF by X. § Studies in literature have reported FF of individual twins to differ by up to two‐fold15, 16. As shown in Table 5, the pattern of fetal fraction estimates seen in the analysis was consistent with what would be expected and demonstrates that the fetal fraction contribution per fetus is not necessarily equal. Interestingly, in one outlier case of a male fetus with trisomy 21 and a female euploid fetus, the fetal fractions by X and chromosome 21 were not similar, at 11.3% and 4.8%, respectively. This pregnancy started as a triplet gestation with demise of one fetus, and therefore it can be speculated that the demised triplet was male, leading to the increased X chromosome‐based fetal fraction estimate.

euploid fetus, the fetal fractions by X and chromosome 21 were not similar, at 11.3% and 4.8%, respectively. This pregnancy started as a triplet gestation with demise of one fetus, and therefore it can be speculated that the demised triplet was male, leading to the increased X chromosome‐based fetal fraction estimate. Clinical Study B test indications Indications for NIPT were reviewed for the 479 reported cases in Clinical Study B, with 443 (92.5%) cases listing one or more indications on the TRF. Amongst a variety of indications (not all listed here), the most common was AMA in 63.3% (303/479), followed by abnormal ultrasound findings in 16.9% (81/479), previous affected pregnancy in 5.4% (26/479) and positive serum screening result in 2.7% (13/479) of cases. Indications were not mutually exclusive and 7.5% (36/479) of cases had more than one indication; all had AMA in addition to one or more of the other indications. Of the six confirmed affected clinical cases, all had AMA as an indication for NIPT, one case also had an ultrasound abnormality and one case also had abnormal ultrasound findings, a positive serum screening result and a previous affected pregnancy. For the two aneuploidy detected/suspected cases with unconfirmed clinical outcomes, clinical indications for testing were AMA in one case and abnormal ultrasound findings in the other.

nd abnormality and one case also had abnormal ultrasound findings, a positive serum screening result and a previous affected pregnancy. For the two aneuploidy detected/suspected cases with unconfirmed clinical outcomes, clinical indications for testing were AMA in one case and abnormal ultrasound findings in the other. Evaluation of the gestational age at which patients requested NIPT revealed that women with AMA or a previous affected pregnancy as the test indication predominantly underwent NIPT in the first trimester (Figure 2). In contrast, women with a positive serum screening result or an abnormal ultrasound finding as the test indication most frequently underwent NIPT in the second trimester. Figure 2 Proportion of cases with twin pregnancy undergoing non‐invasive prenatal testing indicated by advanced maternal age (AMA), positive serum screening result, abnormal ultrasound findings (US) or a previous (Prev.) affected pregnancy, presenting in the first (10–13 weeks; ), second (14–27 weeks; ) or third (28–40 weeks; ) trimester.

of cases with twin pregnancy undergoing non‐invasive prenatal testing indicated by advanced maternal age (AMA), positive serum screening result, abnormal ultrasound findings (US) or a previous (Prev.) affected pregnancy, presenting in the first (10–13 weeks; ), second (14–27 weeks; ) or third (28–40 weeks; ) trimester. UOG-15964-FIG-0002-bDISCUSSION Although there is considerable evidence for robust NIPT performance in singleton pregnancies1, 17, 22, 23, 24, 25, there is still relatively little published about its performance in twins8, 9, 10, 26. Here, we demonstrate the feasibility and clinical application of a cfDNA WGS‐based NIPT for fetal aneuploidy screening in twin pregnancies. Standard serum screening approaches have lower detection rates in twin pregnancies when compared with singletons, high false‐positive rates27 and often cannot provide a result for trisomies 18 or 13. Therefore, there is a need for an accurate non‐invasive method for fetal aneuploidy detection of trisomies 21, 18 and 13 in twin pregnancies. This is a particularly desirable option for patients who are risk averse, such as ART patients, for whom fear of procedure‐related loss is heightened due to difficulties achieving pregnancy. Over half of the cases in Clinical Study A were ART pregnancies; information on the proportion of ART pregnancies in Clinical Study B was unavailable. In Clinical Study A, the method identified correctly all four trisomic twin pregnancies with no false positive or negative. In Clinical Study B, evaluation of the 479 reported clinical twin samples revealed nine with aneuploidy detected or suspected, no reported false positive in the aneuploidy detected samples, one reported false positive in aneuploidy suspected samples and no reported false negative.

es with no false positive or negative. In Clinical Study B, evaluation of the 479 reported clinical twin samples revealed nine with aneuploidy detected or suspected, no reported false positive in the aneuploidy detected samples, one reported false positive in aneuploidy suspected samples and no reported false negative. For Clinical Study B, evaluation of population demographics and indications for testing provided insights into the patient population choosing NIPT. In this population, 68.4% of samples originated from women in their first trimester of pregnancy and the average maternal age was 35.5 years. This is consistent with singleton samples received by the laboratory during the same study period (data not shown). The finding that 92.5% of clinical patients had at least one high‐risk indication on the TRF supports the view that, at present at this laboratory, NIPT in twin pregnancies is being utilized primarily by patients at high risk for fetal aneuploidy.

eceived by the laboratory during the same study period (data not shown). The finding that 92.5% of clinical patients had at least one high‐risk indication on the TRF supports the view that, at present at this laboratory, NIPT in twin pregnancies is being utilized primarily by patients at high risk for fetal aneuploidy. This is the first known study detailing NIPT reporting of fetal sex information for twins. In this clinical population, the majority (87.5%) of patients requested fetal sex information in addition to aneuploidy screening results. Combining results from Clinical Studies A (Table 2) and B (Table 4) revealed a detection rate for chromosome Y of 99.5% (206/207; 91/91 + 115/116) and specificity of 98.4% (60/61; 24/24 + 36/37). Thus, although NIPT has a high degree of accuracy, discordant results, including inaccurate fetal sex prediction, can occur. For cases with discordance in fetal sex results between NIPT and ultrasound, there are several steps that clinicians can consider before invasive diagnostic testing is carried out22. These include assessment of maternal history, checking for a possible demised twin and performance of a detailed ultrasound exam. The benefits and limitations of fetal sex prediction by NIPT should be explained clearly to all patients before consent for testing22.

er before invasive diagnostic testing is carried out22. These include assessment of maternal history, checking for a possible demised twin and performance of a detailed ultrasound exam. The benefits and limitations of fetal sex prediction by NIPT should be explained clearly to all patients before consent for testing22. One of the principal areas of discussion surrounding the application of NIPT in twin pregnancies is that of fetal fraction. Although the total fetal fraction of twin pregnancies might be higher than that of singletons10, the individual contribution from each twin is generally lower than that of a singleton8, 9; however, there may be exceptions. In our study, the average combined fetal fraction of both twins (16.1%) was higher than determined previously for singletons over similar gestational‐age ranges (12.6%; P = 0.001)21, and higher still than the average fetal fraction for a single twin (7.8%; P < 0.0001), consistent with the previously mentioned studies. Rava et al. described fetal‐fraction thresholds for this WGS‐based NIPT approach, demonstrating the capacity for this approach to detect aneuploidy at low fetal fractions in singleton pregnancies21. Fetal fraction estimates using chromosomes X and 21 can provide insight into the fetal fraction contribution per fetus and also allow independent checks of the value for samples confirmed to be trisomy 21 with at least one male fetus, as outlined in Table 5. Here, the technical cancellation rate for twin samples was 0%. In contrast, studies using different NIPT approaches, which apply a fetal‐fraction cut‐off because of reduced sensitivity at low fetal fractions, reported first‐draw technical failure rates of 5.6% and 7.3%8, 9. Importantly, no trisomic pregnancies were missed in Clinical Study A, no false negatives were reported for Clinical Study B and no false positives were reported for aneuploidy detected findings in either study. In contrast, three other studies in twin pregnancies reported at least one false negative8, 9, 26, despite two of the studies applying a 4% fetal‐fraction threshold (applied to the lower of the two fetal fraction values) for reporting8, 9. Here, fetal fraction estimates using chromosomes X and 21 were used for study analysis purposes only, and these estimates were not reported to the patient. However, we are developing better methods to assess reliably fetal fraction in response to physician interest.

of the two fetal fraction values) for reporting8, 9. Here, fetal fraction estimates using chromosomes X and 21 were used for study analysis purposes only, and these estimates were not reported to the patient. However, we are developing better methods to assess reliably fetal fraction in response to physician interest. The work reported here has some limitations. Like other published twin studies8, 9, 10, 26, the number of affected pregnancies was small and the majority were trisomy 21. This precluded determination of detection rates for trisomies 13 and 18. Although there were no positive calls for trisomy 13, this study does allow an evaluation of the specificity and false‐positive frequency for trisomy 13. It is also important to note that we had no reports of false‐negative calls for trisomy 13. These studies also allow determination of the overall observed trisomy false‐positive frequency: 0% in Clinical Study A and 0.2% in Clinical Study B. These values are in line with the 0–0.3% combined false‐positive rate described for singletons1, 17. Another limitation was incomplete clinical outcomes, with aneuploidy outcome information available for only 35.7% (171/479) of cases in Clinical Study B; obtaining outcomes remains a challenge for clinical laboratories. Also, neither zygosity nor chorionicity information was available for the majority of patients in Clinical Study B.

ion was incomplete clinical outcomes, with aneuploidy outcome information available for only 35.7% (171/479) of cases in Clinical Study B; obtaining outcomes remains a challenge for clinical laboratories. Also, neither zygosity nor chorionicity information was available for the majority of patients in Clinical Study B. In this study, we successfully demonstrated detection of fetal trisomies and the Y chromosome by NIPT in twin pregnancies. The detection rate for trisomy 21 in twins appears to be in line with that in singletons. The limited number of affected cases for other trisomies precluded conclusive determination of those detection rates. In summary, the findings reported here support the view that cfDNA WGS‐based NIPT performs well in twin pregnancies, with overall very low false‐positive frequencies. DISCLOSURES L. Fosler, P. Winters, K. W. Jones, K. J. Curnow, A. J. Sehnert and S. Bhatt are, or were, employees of, and hold equity in, Illumina. L. D. Platt is a paid consultant for Illumina. This study was funded by Illumina.

INTRODUCTION Pregnant women at high risk for fetal chromosomal abnormality based on findings at sonographic examination are offered prenatal genetic testing. Genetic evaluation enables determination of the etiology of the ultrasound anomaly, overall fetal prognosis and risk of recurrence in future pregnancies. The information obtained could facilitate prospective parents' reproductive decision‐making when confronted with the choice between terminating pregnancy or continuing pregnancy and preparing themselves for the birth of a child with a physical and/or intellectual disability.

nd risk of recurrence in future pregnancies. The information obtained could facilitate prospective parents' reproductive decision‐making when confronted with the choice between terminating pregnancy or continuing pregnancy and preparing themselves for the birth of a child with a physical and/or intellectual disability. In cases with ultrasound anomalies, fetal genotyping is usually performed by quantitative fluorescent polymerase chain reaction (QF‐PCR) of fetal DNA for rapid diagnosis of (an)euploidy of chromosomes 13, 18, 21 and the sex chromosomes. If QF‐PCR results are normal, microscopic G‐band karyotyping has long been the standard approach to detect chromosomal aberrations at a resolution of 5–10 megabases (Mb). At present, chromosomal microarray analysis is recommended instead of, or as an adjunct to, conventional karyotyping, as it allows genome‐wide analysis at a much higher resolution (up to 1 kilobase (kb)), revealing clinically significant unbalanced submicroscopic aberrations1, 2, 3. However, these techniques all require fetal tissue obtained by an invasive diagnostic procedure, i.e. chorionic villus sampling or amniocentesis. For a considerable number of pregnant women at high risk for fetal chromosomal abnormality, the associated procedure‐related risk of miscarriage is a reason to decline invasive diagnostic testing.

ues all require fetal tissue obtained by an invasive diagnostic procedure, i.e. chorionic villus sampling or amniocentesis. For a considerable number of pregnant women at high risk for fetal chromosomal abnormality, the associated procedure‐related risk of miscarriage is a reason to decline invasive diagnostic testing. The development of non‐invasive prenatal testing (NIPT) by massively parallel sequencing (MPS) of cell‐free DNA in maternal plasma provides an alternative for the detection of fetal genetic aberrations with the increased safety of performing maternal blood sampling instead of an invasive diagnostic procedure. NIPT detects common whole‐chromosome aneuploidy such as trisomy 21 (T21, Down syndrome), trisomy 18 (T18, Edwards syndrome) and trisomy 13 (T13, Patau syndrome) with high sensitivity and specificity4. The non‐invasive detection of subchromosomal copy‐number variants (CNVs), i.e. microdeletions and microduplications, at a resolution comparable with that of microarray analysis has been shown to be feasible5. However, routine implementation of the latter technique is hampered by its requirements for significantly deeper sequencing, which is costly6. Furthermore, extensive validation is needed to determine accurately its detection rate and false‐positive rate, which is difficult owing to the low prevalence of specific genetic aberrations. Nonetheless, some pregnant women at high risk for fetal chromosomal abnormality based on findings at sonographic examination already consider NIPT an acceptable alternative to invasive diagnostic testing.

ction rate and false‐positive rate, which is difficult owing to the low prevalence of specific genetic aberrations. Nonetheless, some pregnant women at high risk for fetal chromosomal abnormality based on findings at sonographic examination already consider NIPT an acceptable alternative to invasive diagnostic testing. This study aimed to evaluate the application of NIPT as an alternative to invasive diagnostic testing for the detection of fetal genetic aberrations in pregnancies with ultrasound anomalies. METHODS Study design and subjects This was a retrospective study of all pregnant women at high risk for fetal chromosomal abnormality based on findings at sonographic examination, who underwent NIPT as an alternative to fetal genotyping by QF‐PCR and microarray analysis at prenatal diagnostic centers linked to the Network for Prenatal Diagnosis Nijmegen, between April 2014 and November 2015. Ethical approval was granted by the medical ethics committee of the Radboud University Medical Center, Nijmegen, The Netherlands.

alternative to fetal genotyping by QF‐PCR and microarray analysis at prenatal diagnostic centers linked to the Network for Prenatal Diagnosis Nijmegen, between April 2014 and November 2015. Ethical approval was granted by the medical ethics committee of the Radboud University Medical Center, Nijmegen, The Netherlands. Counseling Pregnant women at high risk for fetal chromosomal abnormality based on sonographic examination were offered prenatal genetic testing. As part of the standard care offered in the prenatal diagnostic centers linked to the Network for Prenatal Diagnosis Nijmegen, all prospective parents included in the cohort received thorough pretest counseling on the test characteristics and potential benefits and disadvantages of different modalities for prenatal genetic testing. Invasive diagnostic testing was offered as the first‐tier genetic test, utilizing both QF‐PCR and chromosomal microarray analysis, thereby providing maximum information and a definitive diagnosis. Counseling included information on the invasive diagnostic procedure needed to obtain fetal tissue and the associated procedure‐related risk of miscarriage, i.e. 0.5% for chorionic villus sampling and 0.3% for amniocentesis.

omosomal microarray analysis, thereby providing maximum information and a definitive diagnosis. Counseling included information on the invasive diagnostic procedure needed to obtain fetal tissue and the associated procedure‐related risk of miscarriage, i.e. 0.5% for chorionic villus sampling and 0.3% for amniocentesis. Counseling on NIPT included information on the test being validated for the detection of only T21, T18 and T13, thus offering limited information compared with invasive diagnostic testing. Pregnant women and their partners were informed that other potentially clinically relevant abnormal test results besides T21, T18 and T13 would be discussed with them. Furthermore, all abnormal test results would have to be confirmed by invasive diagnostic testing, especially when the results would influence management of their pregnancy. Those who decided to have NIPT as an alternative to fetal genotyping by QF‐PCR and microarray analysis provided written informed consent for the application of NIPT as a first‐tier genetic test.

ve to be confirmed by invasive diagnostic testing, especially when the results would influence management of their pregnancy. Those who decided to have NIPT as an alternative to fetal genotyping by QF‐PCR and microarray analysis provided written informed consent for the application of NIPT as a first‐tier genetic test. Genetic testing For NIPT, maternal blood samples were obtained at one of five diagnostic centers linked to the Network for Prenatal Diagnosis Nijmegen. All samples were then processed and analyzed at the Department of Human Genetics of the Radboud University Medical Center in Nijmegen, as described previously7, 8. Briefly, samples underwent centrifugation for plasma separation, followed by manual extraction and MPS of cell‐free DNA from maternal plasma. MPS entailed either single‐end 50‐bp sequencing on a SOLiD 5500 XL Genetic Analyzer (Life Technologies, Foster City, CA, USA) or single‐end 75‐bp sequencing on a NextSeq 500 desktop sequencer (Illumina, San Diego, CA, USA). After sequencing, reads were mapped to the hg19 reference genome. Samples were tested for fetal chromosomal aberrations using WISECONDOR, a within‐sample comparison algorithm using GC‐corrected 1‐Mb genomic bins9, 10. With this algorithm, both whole‐chromosome, as well as partial, aneuploidy can be detected. A whole‐chromosome fetal trisomy or monosomy is considered detected when at least 50% of the bins of a chromosome are aberrant. Using low sequence coverage (0.1–0.5×), aberrations smaller than 20 Mb cannot be detected reliably owing to natural fluctuations of read‐depth frequencies along the genome. Smaller aberrations are, therefore, considered only if visual inspection of the plots strongly suggests a fetal genetic aberration. Currently, in Dutch laboratories, fetal gender and sex chromosomal aberrations are not analyzed when performing NIPT. Furthermore, fetal fraction is not determined. Instead, it is assumed that the blood sample contains sufficient cell‐free fetal DNA to allow the identification of fetal chromosomal aberrations.

rration. Currently, in Dutch laboratories, fetal gender and sex chromosomal aberrations are not analyzed when performing NIPT. Furthermore, fetal fraction is not determined. Instead, it is assumed that the blood sample contains sufficient cell‐free fetal DNA to allow the identification of fetal chromosomal aberrations. If prenatal or postnatal diagnostic testing was performed, QF‐PCR analysis was carried out according to standard procedures. In cases of normal QF‐PCR results, genome‐wide high‐resolution microarray analysis was carried out according to the manufacturer's specifications on an Affymetrix CytoScan HD array platform (Affymetrix, Inc., Santa Clara, CA, USA). Cut‐offs for detection criteria for CNVs were set at 20 kb for gains, 10 kb for losses and 1250 kb for regions of homozygosity11. Where applicable, parental DNA was analyzed to facilitate the interpretation of abnormal microarray results. If CNVs were inherited from a healthy parent, i.e. parents without physical or neurodevelopmental anomalies, they were classified as most probably benign. Targeted DNA testing for monogenic diseases was performed in adjunct to microarray analysis if indicated.

cilitate the interpretation of abnormal microarray results. If CNVs were inherited from a healthy parent, i.e. parents without physical or neurodevelopmental anomalies, they were classified as most probably benign. Targeted DNA testing for monogenic diseases was performed in adjunct to microarray analysis if indicated. Data collection Medical records were reviewed to extract the following data: maternal age, maternal weight, gravidity, parity, sonographic findings that indicated the need for prenatal genetic testing, gestational age at time of blood sampling for NIPT, results of NIPT, (additional) findings of any follow‐up sonographic examination, results of any diagnostic genetic testing and pregnancy outcome, including findings at the newborn examination. Pathogenic aberrations detected by genetic testing were classified as causative findings, unexpected diagnoses or susceptibility loci for neurodevelopmental disorders based on their relationship with findings of the sonographic and/or newborn examination. Aberrations found unintentionally in prospective parents were classified as incidental findings12.

genetic testing were classified as causative findings, unexpected diagnoses or susceptibility loci for neurodevelopmental disorders based on their relationship with findings of the sonographic and/or newborn examination. Aberrations found unintentionally in prospective parents were classified as incidental findings12. RESULTS Between April 2014 and November 2015, prenatal genetic testing was performed in 590 pregnant women at high risk for fetal chromosomal abnormality based on sonographic examination. NIPT was applied as the first‐tier test for the detection of genetic aberrations in 251 (42.5%) pregnancies. Of these, 230 (91.6%) were singleton and 21 (8.4%) were multiple pregnancies. In all multiple pregnancies, ultrasound anomalies were present in only one of the fetuses when genetic testing was first indicated, thus also when NIPT was performed. Characteristics of the pregnant women included in the cohort are shown in Table 1. Table 1 Characteristics of 251 pregnant women with abnormal findings at sonographic examination who had non‐invasive prenatal testing (NIPT) as first‐tier genetic test Characteristic Value Age (years) 31 (17–44) Maternal weight at NIPT (kg) 69 (44–123) Singleton pregnancy 230 (91.6) Multiple pregnancy 21 (8.4) Primigravid 94 (37.5) Multigravid 157 (62.5) Nulliparous 119 (47.4) Parous 132 (52.6) GA at NIPT (weeks)* 20 (10–34) Data are given as median (range) or n (%). * Gestational age (GA) had bimodal distribution, with peaks at 12–13 weeks and at 20–21 weeks.

Characteristic Value Age (years) 31 (17–44) Maternal weight at NIPT (kg) 69 (44–123) Singleton pregnancy 230 (91.6) Multiple pregnancy 21 (8.4) Primigravid 94 (37.5) Multigravid 157 (62.5) Nulliparous 119 (47.4) Parous 132 (52.6) GA at NIPT (weeks)* 20 (10–34) Data are given as median (range) or n (%). * Gestational age (GA) had bimodal distribution, with peaks at 12–13 weeks and at 20–21 weeks. An overview of the findings from NIPT and diagnostic genetic testing in pregnancies with sonographic anomalies is shown in Figure 1, Tables 2, 3, 4, Table S1 and (showing multiple gestations only) Table S2. NIPT results were normal in 224 (89.2%) pregnancies, inconclusive in one (0.4%) and abnormal in 26 (10.4%).

* Gestational age (GA) had bimodal distribution, with peaks at 12–13 weeks and at 20–21 weeks. An overview of the findings from NIPT and diagnostic genetic testing in pregnancies with sonographic anomalies is shown in Figure 1, Tables 2, 3, 4, Table S1 and (showing multiple gestations only) Table S2. NIPT results were normal in 224 (89.2%) pregnancies, inconclusive in one (0.4%) and abnormal in 26 (10.4%). Figure 1 Flowchart of results of non‐invasive prenatal testing (NIPT) in 251 pregnancies with ultrasound anomaly. NIPT was by massively parallel sequencing of cell‐free DNA in maternal plasma. Diagnostic genetic testing was by quantitative fluorescent polymerase chain reaction, karyotyping, microarray analysis or targeted DNA testing of fetal or maternal tissue. Case 1: genetic testing indicated by polyhydramnios, NIPT did not meet quality criteria, however, visual inspection of plots repeatedly showed suspect chromosome 22 profile. Mother had mild intellectual disability, shortened palate, mild hearing impairment, common variable immunodeficiency. Maternal array was 22q11.21(18,970,562‐21,465,660)×1, ∼2.5 megabases (Mb). No further genetic testing. Resulted in live birth with no congenital anomaly. Postnatal microarray of newborn revealed 22q11.21(18,648,867‐21,798,908)×1 mat, ∼2.8 Mb. Case 2: multiple pregnancy, genetic testing indicated by multiple structural anomalies (intrauterine growth restriction, holoprosencephaly, omphalocele, megacystis with bilateral hydro‐nephrosis) in one fetus. No structural anomaly was observed in other fetus. NIPT showed T21. Follow‐up ultrasound at 17 weeks' gestation revealed atrioventricular septal defect and absent nasal bone in fetus without previously detected anomalies. Amniocentesis and genetic testing showed 47,XY + 21 and 47,XY + 13. Pregnancy was terminated. Cases 3–13 summarized in Tables 2 and 3. CNV, copy number variant; del, deletion; dup, duplication; kb, kilobases; MOPD1, microcephaly osteodysplastic primordial dwarfism Type I; T13, trisomy 13 (Patau syndrome); T16, trisomy 16; T18, trisomy 18 (Edwards syndrome); T21, trisomy 21 (Down syndrome); UPD(16)mat, maternal uniparental disomy of chromosome 16.

d 3. CNV, copy number variant; del, deletion; dup, duplication; kb, kilobases; MOPD1, microcephaly osteodysplastic primordial dwarfism Type I; T13, trisomy 13 (Patau syndrome); T16, trisomy 16; T18, trisomy 18 (Edwards syndrome); T21, trisomy 21 (Down syndrome); UPD(16)mat, maternal uniparental disomy of chromosome 16. UOG-17228-FIG-0001-bTable 2 Overview of cases with abnormal non‐invasive prenatal test (NIPT) result other than trisomy 21, 18 or 13, in 251 pregnancies with ultrasound anomaly Case Indication for genetic testing NIPT Diagnostic genetic testing Outcome Classification of aberration Timing Result Timing Test Result 3 IUGR 15 weeks T16 16 weeks QF‐PCR, karyotype, FISH 46,XX 46,XX D16Z3 × 2 Preterm LB with no congenital anomaly, extreme dysmaturity, perinatal death Causative Postnatal Microarray arr 16p13.3p13.2(89,561‐8,914,906) × 2 hmz = UPD (16)mat* 4 NT 4.7 mm 13 weeks Gain of 8p† 16 weeks Karyotype, microarray 46,XX,del(8)(p23.3p23.1), dup(8)(p23.1p11.21) arr 8p23.3p23.1(158,049‐6,976,182) × 1 dn, 6.8 Mb, 8p23.1p11.21(11,936,001‐40,905,009) × 3 dn, 29 Mb TOP Causative 5 IUGR 21 weeks Gain of 21q‡ 22 weeks Maternal microarray arr 21q22.11(33,522,970‐33,889,304) × 3, ∼370 kb (6 genes) LB with no congenital anomaly Incidental 6 IUGR 21 weeks Gain of 18p§ NP LB, two small VSDs (resolved spontaneously), mild dysmorphisms * Maternal uniparental disomy of chromosome 16, most likely due to ‘trisomy 16 rescue’; placental material not available for testing. † Software analysis showed 30‐Mb gain of 8p; visual inspection of plots showed also a (smaller) 8p terminal loss.

arr 8p23.3p23.1(158,049‐6,976,182) × 1 dn, 6.8 Mb, 8p23.1p11.21(11,936,001‐40,905,009) × 3 dn, 29 Mb TOP Causative 5 IUGR 21 weeks Gain of 21q‡ 22 weeks Maternal microarray arr 21q22.11(33,522,970‐33,889,304) × 3, ∼370 kb (6 genes) LB with no congenital anomaly Incidental 6 IUGR 21 weeks Gain of 18p§ NP LB, two small VSDs (resolved spontaneously), mild dysmorphisms * Maternal uniparental disomy of chromosome 16, most likely due to ‘trisomy 16 rescue’; placental material not available for testing. † Software analysis showed 30‐Mb gain of 8p; visual inspection of plots showed also a (smaller) 8p terminal loss. ‡ Software analysis showed 10‐Mb gain of 21q; visual inspection of plots showed relatively high region‐specific Z‐score, therefore, maternal origin was suspected; consultation with clinical geneticist revealed no maternal dysmorphic features or physical or neurodevelopmental anomaly; copy number variant therefore classified as most probably benign. § Software analysis showed 15‐Mb gain of 18p. FISH, fluorescent in‐situ hybridization; IUGR, intrauterine growth restriction; kb, kilobases; LB, live birth; Mb, megabases; NP, not performed; NT, nuchal translucency thickness; QF‐PCR, quantitative fluorescent polymerase chain reaction; T16, trisomy 16; TOP, termination of pregnancy; VSD, ventricular septal defect. Table 3 Overview of cases with clinically relevant aberration revealed by diagnostic genetic testing after normal non‐invasive prenatal test (NIPT) result in 251 pregnancies with ultrasound (US) anomaly

FISH, fluorescent in‐situ hybridization; IUGR, intrauterine growth restriction; kb, kilobases; LB, live birth; Mb, megabases; NP, not performed; NT, nuchal translucency thickness; QF‐PCR, quantitative fluorescent polymerase chain reaction; T16, trisomy 16; TOP, termination of pregnancy; VSD, ventricular septal defect. Table 3 Overview of cases with clinically relevant aberration revealed by diagnostic genetic testing after normal non‐invasive prenatal test (NIPT) result in 251 pregnancies with ultrasound (US) anomaly Case Indication for genetic testing Timing Diagnostic genetic testing Outcome Classification of aberration Timing Test Result 7 MCA 14 weeks 17 weeks QF‐PCR 47,XY + 13 IUFD (US at 17 weeks) Causative 8 Hydrops fetalis 12 weeks Postnatal QF‐PCR 45,X IUFD (US at 18 weeks) Causative 9 NT 3.9 mm 21 weeks 32 weeks Microarray 14q32.2q32.33(101,220,548‐105,080,719) × 1 dn, 3.9 Mb Unilateral hydrothorax (US at 32 weeks), anal atresia, some mild dysmorphisms Causative 10 IUGR 23 weeks Postnatal Microarray 4p16.3p15.33(68,346‐14,875,532) × 1 dn, 14.8 Mb(Wolf–Hirschhorn syndrome) 10q26.11q26.3(120,145,796‐135,427,144) × 3 dn, 15.3 Mb Severe dysmaturity, microcephaly, some dysmorphisms Causative 11 MCA 30 weeks Postnatal DNA Homozygous pathogenic mutation in RNU4ATAC gene (LIT1) (MOPD1) Severe dysmaturity, microcephaly, severe intracerebral abnormalities, VSD, several dysmorphisms, bilateral rocker bottom feet Causative 12 Omphalocele 14 weeks Postnatal Microarray 16p11.2(29,567,296‐30,178,000) × 3 dn, 610 kb(susceptibility locus)

atal DNA Homozygous pathogenic mutation in RNU4ATAC gene (LIT1) (MOPD1) Severe dysmaturity, microcephaly, severe intracerebral abnormalities, VSD, several dysmorphisms, bilateral rocker bottom feet Causative 12 Omphalocele 14 weeks Postnatal Microarray 16p11.2(29,567,296‐30,178,000) × 3 dn, 610 kb(susceptibility locus) DNA hypomethylation in KCNQ1OT1 gene (Beckwith–Wiedemann syndrome) Omphalocele, unilateral duplex collecting system and ureterocele (US at 18 weeks), earlobe creases Susceptibility locus; causative 13 Echogenic bowel 21 weeks 23 weeks* DNA Homozygous pathogenic mutation in CFTR gene (deltaF508) (cystic fibrosis) TOP Causative * Parental carrier screening confirmed presence of maternal and paternal pathogenic mutations in CFTR gene (deltaF508). IUFD, intrauterine fetal demise; IUGR, intrauterine growth restriction; kb, kilobases; Mb, megabases; MCA, multiple congenital anomalies; MOPD1, microcephaly osteodysplastic primordial dwarfism Type I; NT, nuchal translucency thickness; QF‐PCR, quantitative fluorescent polymerase chain reaction; TOP, termination of pregnancy; VSD, ventricular septal defect. Table 4 Diagnostic yield of non‐invasive prenatal testing (NIPT) and invasive diagnostic genetic testing by quantitative fluorescent polymerase chain reaction (QF‐PCR) and chromosomal microarray analysis in 251 pregnancies with ultrasound anomaly Indication for genetic testing Diagnostic yield NIPT QF‐PCR and microarray analysis* Multiple structural anomalies  7/13 (53.8)  8/13 (61.5) Structural anomaly 4/57 (7.0) 5/57 (8.8)†

Table 4 Diagnostic yield of non‐invasive prenatal testing (NIPT) and invasive diagnostic genetic testing by quantitative fluorescent polymerase chain reaction (QF‐PCR) and chromosomal microarray analysis in 251 pregnancies with ultrasound anomaly Indication for genetic testing Diagnostic yield NIPT QF‐PCR and microarray analysis* Multiple structural anomalies  7/13 (53.8)  8/13 (61.5) Structural anomaly 4/57 (7.0) 5/57 (8.8)† NT ≥ 3.5 mm  9/58 (15.5) 10/58 (17.2) Sonographic marker 2/73 (2.7) 2/73 (2.7) IUGR  4/40 (10.0)  5/40 (12.5) Other 0/10 (0.0)  2/10 (20.0)‡ Total  26/251 (10.4)  32/251 (12.7) Data are given as n/N (%). * Yield of prenatal genetic testing if invasive diagnostic testing was performed in all pregnancies in cohort. Parentally inherited copy number variants < 1 megabase (Mb) classified as most probably benign and not clinically relevant and monogenic aberrations that would only have been detected by targeted DNA testing were not taken into consideration. † Including Case 12: normal NIPT result, in which 16p11.2 ∼ 610 kilobase gain (susceptibility locus) would have been detected by chromosomal microarray analysis, but (clinically relevant) hypomethylation of KCNQ1OT1 gene resulting in Beckwith–Wiedemann syndrome would not. ‡ Including Case 1: inconclusive NIPT result in which 22q11.21 ∼ 2.8 Mb deletion would have been detected by chromosomal microarray analysis despite maternal carrier status. IUGR, intrauterine growth restriction; NT, nuchal translucency thickness.

† Including Case 12: normal NIPT result, in which 16p11.2 ∼ 610 kilobase gain (susceptibility locus) would have been detected by chromosomal microarray analysis, but (clinically relevant) hypomethylation of KCNQ1OT1 gene resulting in Beckwith–Wiedemann syndrome would not. ‡ Including Case 1: inconclusive NIPT result in which 22q11.21 ∼ 2.8 Mb deletion would have been detected by chromosomal microarray analysis despite maternal carrier status. IUGR, intrauterine growth restriction; NT, nuchal translucency thickness. Abnormal NIPT result The majority of genetic aberrations detected by NIPT were whole‐chromosome aneuploidies: 13 cases of T21, six cases of T18, three cases of T13 and one case of trisomy 16. Follow‐up diagnostic genetic testing was performed in 21 of these 23 pregnancies.

IUGR, intrauterine growth restriction; NT, nuchal translucency thickness. Abnormal NIPT result The majority of genetic aberrations detected by NIPT were whole‐chromosome aneuploidies: 13 cases of T21, six cases of T18, three cases of T13 and one case of trisomy 16. Follow‐up diagnostic genetic testing was performed in 21 of these 23 pregnancies. In two cases, the findings of NIPT and diagnostic genetic testing were discordant. In one specific case, prenatal testing was indicated for genetic evaluation of the etiology of multiple structural anomalies present in one fetus of a diamniotic twin pregnancy. No structural anomaly was observed in the other fetus when NIPT was performed at 12 + 5 weeks' gestation. NIPT was abnormal, indicating the presence of T21. Follow‐up sonographic examination at 17 weeks' gestation revealed an atrioventricular septal defect and absent nasal bone, suggestive of T21, in the fetus without previously detected anomalies. QF‐PCR analysis of fetal tissue obtained by amniocentesis confirmed the presence of T21 (detected by NIPT) in this fetus, but also detected T13 (not detected by NIPT) in the other fetus. When NIPT was repeated for research purposes at 17 + 5 weeks' gestation, both aneuploidies were detected. The pregnancy was terminated.

‐PCR analysis of fetal tissue obtained by amniocentesis confirmed the presence of T21 (detected by NIPT) in this fetus, but also detected T13 (not detected by NIPT) in the other fetus. When NIPT was repeated for research purposes at 17 + 5 weeks' gestation, both aneuploidies were detected. The pregnancy was terminated. In the second case with discordant results, NIPT revealed fetal trisomy 16, or mosaic thereof, which could be confirmed neither by karyotyping of amniotic fluid nor by fluorescent in‐situ hybridization analysis using a probe specific for chromosome 16 (D16Z3). However, postnatal microarray analysis revealed the presence of maternal uniparental disomy of chromosome 16, most probably owing to so‐called ‘trisomy 16 rescue’. This, in combination with the observed intrauterine growth restriction and preterm birth, led to a suspected diagnosis of confined placental mosaicism. Unfortunately, placental material was not available for testing (Table 2).

al uniparental disomy of chromosome 16, most probably owing to so‐called ‘trisomy 16 rescue’. This, in combination with the observed intrauterine growth restriction and preterm birth, led to a suspected diagnosis of confined placental mosaicism. Unfortunately, placental material was not available for testing (Table 2). In two cases, abnormal NIPT results were not confirmed by diagnostic genetic testing. In one case, a diamniotic twin pregnancy, prenatal testing was indicated because of a high risk for fetal aneuploidy based on prenatal screening by first‐trimester combined test (risk: T21, 1:6; T18, 1:22; T13, 1:45), with a nuchal translucency thickness of 3.8 mm in one fetus. The risk for fetal aneuploidy in the other fetus was not elevated. NIPT was performed at 12 + 6 weeks' gestation and indicated the presence of T21. Sonographic follow‐up diagnosed intrauterine demise of the affected twin at 14 weeks. Amniocentesis was performed only on the fetus with normal nuchal translucency. QF‐PCR results were normal. In the second case with an abnormal NIPT result not confirmed by diagnostic genetic testing, prenatal testing was indicated by the presence of multiple structural anomalies (semilobar holoprosencephaly, atrioventricular septal defect with hypoplastic left heart) and severe intrauterine growth restriction. Although fetal T18, detected by NIPT, fitted the phenotype, consent for prenatal or postnatal diagnostic testing was not given. The pregnancy ended in intrauterine fetal demise.

nomalies (semilobar holoprosencephaly, atrioventricular septal defect with hypoplastic left heart) and severe intrauterine growth restriction. Although fetal T18, detected by NIPT, fitted the phenotype, consent for prenatal or postnatal diagnostic testing was not given. The pregnancy ended in intrauterine fetal demise. Subchromosomal aberrations were detected by NIPT in three other pregnancies (Table 2). Owing to the relatively high region‐specific Z‐score in one case, a maternal origin was suspected, which was subsequently confirmed via microarray analysis of the mother's genomic DNA. Consultation with a clinical geneticist revealed no maternal dysmorphic features, or physical or neurodevelopmental anomaly, therefore the CNV was classified as most probably benign. In another case, NIPT detected accurately an inverted duplication deletion of chromosome 8p. In the third case, a gain of chromosome 18p was detected, however, consent for prenatal or postnatal diagnostic confirmation was not given. Newborn examination revealed mild dysmorphic features and two small ventricular septal defects that resolved spontaneously.

ately an inverted duplication deletion of chromosome 8p. In the third case, a gain of chromosome 18p was detected, however, consent for prenatal or postnatal diagnostic confirmation was not given. Newborn examination revealed mild dysmorphic features and two small ventricular septal defects that resolved spontaneously. Normal NIPT result Among the 224 pregnancies with normal NIPT results, (additional) findings at sonographic follow‐up or newborn examination indicated the need for diagnostic genetic testing in 33 (14.7%) cases. Diagnostic testing revealed genetic aberrations in 12 of these pregnancies, seven (3.1%) of which were clinically relevant (Table 3). In all other cases, newborn examination was normal or showed congenital anomalies that did not indicate the need for additional genetic testing.

stic genetic testing in 33 (14.7%) cases. Diagnostic testing revealed genetic aberrations in 12 of these pregnancies, seven (3.1%) of which were clinically relevant (Table 3). In all other cases, newborn examination was normal or showed congenital anomalies that did not indicate the need for additional genetic testing. Aside from the previously described case of a diamniotic twin pregnancy with an abnormal NIPT result indicating T21 but missing T13, NIPT did not detect two other common whole‐chromosome aneuploidies: one case of T13 and one of monosomy X. As sex chromosomal aberrations were not included in the NIPT analysis, the latter aneuploidy cannot be considered a false‐negative result. In two other cases, clinically relevant subchromosomal aberrations were not detected by NIPT: ∼14.8‐Mb 4p16.3p15.33 deletion, associated with Wolf–Hirschhorn syndrome, in combination with ∼15.3‐Mb 10q26.11q26.3 duplication in one case, and a submicroscopic ∼3.9‐Mb 14q32.2q32.33 deletion in another case. Other subchromosomal aberrations detected by postnatal diagnostic testing were parentally inherited CNVs of < 1 Mb. As all were inherited from a healthy parent, these were classified as most probably benign and not clinically relevant. Targeted DNA testing confirmed the presence of monogenic aberrations in three other cases. These monogenic aberrations cannot be detected by techniques based on whole‐genome analysis of CNVs, such as NIPT or microarray analysis.

healthy parent, these were classified as most probably benign and not clinically relevant. Targeted DNA testing confirmed the presence of monogenic aberrations in three other cases. These monogenic aberrations cannot be detected by techniques based on whole‐genome analysis of CNVs, such as NIPT or microarray analysis. If invasive diagnostic testing with QF‐PCR and chromosomal microarray analysis had been performed as the first‐tier genetic test in all pregnancies included in the cohort, genetic aberrations would have been detected in 12.7% instead of 10.4% of all pregnancies, excluding parentally inherited CNVs < 1 Mb as these were most probably benign and not clinically relevant (Table 4). DISCUSSION Our study evaluates the application of NIPT for prenatal genetic testing in pregnancies at high risk for fetal chromosomal abnormality based on findings at sonographic examination. Our findings show that NIPT should not be recommended as an alternative to invasive diagnostic testing for genetic evaluation of the etiology of ultrasound anomalies. Both resolution and sensitivity, or negative predictive value, are inferior to those of the diagnostic genetic test modalities currently available, i.e. conventional G‐band karyotyping and chromosomal microarray analysis.

ive to invasive diagnostic testing for genetic evaluation of the etiology of ultrasound anomalies. Both resolution and sensitivity, or negative predictive value, are inferior to those of the diagnostic genetic test modalities currently available, i.e. conventional G‐band karyotyping and chromosomal microarray analysis. NIPT for the detection of fetal genetic aberrations in pregnancies with ultrasound anomalies yielded 10.4% abnormal results. This is considerably higher than the yield found in pregnancies at high risk for fetal chromosomal abnormality based on a personal or family history of chromosomal abnormality or prenatal screening by first‐trimester combined testing (5.2% abnormal NIPT results in the same timeframe, applying the same laboratory protocol; own unpublished data). This is in accordance with the high prevalence of fetal genetic aberrations in the population studied. Chromosomal and subchromosomal genetic aberrations that can be detected by conventional karyotyping are present in 9–19% of fetuses with single and multiple anomalies, respectively13. Clinically relevant submicroscopic CNVs detected by microarray analysis are found in 5–10% of fetuses with ultrasound anomalies and a normal karyotype1, 2, 3.

omosomal genetic aberrations that can be detected by conventional karyotyping are present in 9–19% of fetuses with single and multiple anomalies, respectively13. Clinically relevant submicroscopic CNVs detected by microarray analysis are found in 5–10% of fetuses with ultrasound anomalies and a normal karyotype1, 2, 3. With the present state of technology, NIPT should be able to detect common whole‐chromosome aneuploidies with high sensitivity and specificity4. However, in this study population, two out of five cases of T13 were not detected by NIPT. Discordant findings could possibly be explained by a low fraction of fetal DNA within these samples, directly affecting the diagnostic reliability of NIPT. Research has shown that the fetal fraction is decreased when T13 is present in the fetus14, 15. Furthermore, in multiple pregnancies the fetal fraction per fetus is reduced, leading to an increased risk of false‐negative results16. Unfortunately, in Dutch laboratories, the fraction of fetal DNA is not at present determined when performing NIPT, therefore no additional information regarding fetal fraction was available.

rmore, in multiple pregnancies the fetal fraction per fetus is reduced, leading to an increased risk of false‐negative results16. Unfortunately, in Dutch laboratories, the fraction of fetal DNA is not at present determined when performing NIPT, therefore no additional information regarding fetal fraction was available. In the presence of sufficient fetal DNA, the resolution and sensitivity of NIPT are limited mainly by sequencing depth6, 17. It is to be expected that, in due time, technological development will enable non‐invasive detection of small pathogenic genetic aberrations, such as microdeletions and microduplications. However, even if the resolution and sensitivity of NIPT were comparable with those of microarray analysis, NIPT cannot be considered a diagnostic test. The present study shows that discordant findings between NIPT and true fetal genotype are unavoidable owing to biological phenomena such as confined placental mosaicism and maternal genetic aberrations.

d sensitivity of NIPT were comparable with those of microarray analysis, NIPT cannot be considered a diagnostic test. The present study shows that discordant findings between NIPT and true fetal genotype are unavoidable owing to biological phenomena such as confined placental mosaicism and maternal genetic aberrations. Although microarray analysis is recommended as the first‐tier test for the detection of fetal genetic aberrations in pregnancies with ultrasound anomalies, for some pregnant women and their partners, NIPT represents an acceptable alternative to invasive diagnostic testing. Prospective parents might evaluate differently the benefits and disadvantages of different test modalities depending on many factors, i.e. the value of the information provided by the test, the intention to terminate or continue pregnancy in case of (relevant) fetal genetic aberrations, and acceptability of the risk of a procedure‐related miscarriage18. Therefore, the choice of a prenatal test should be theirs to make, provided they receive thorough pretest counseling on the characteristics of genetic tests currently available. Pregnant women and their partners should be made fully aware that the information provided by NIPT is inferior to the information provided by invasive diagnostic testing. Furthermore, pregnant women should realize that there is a chance of unlooked‐for findings in the form of maternal CNVs.

c tests currently available. Pregnant women and their partners should be made fully aware that the information provided by NIPT is inferior to the information provided by invasive diagnostic testing. Furthermore, pregnant women should realize that there is a chance of unlooked‐for findings in the form of maternal CNVs. For pregnancies included in this study, fear of a procedure‐related miscarriage and unwillingness to terminate the pregnancy in case of fetal chromosomal abnormality were often noted as reasons for declining invasive diagnostic testing. Unfortunately, the data extracted from medical records were insufficient for further analysis of prospective parents' decision‐making. Prospective studies comparing the utility of invasive diagnostic testing with the utility of NIPT for genetic evaluation of pregnancies with ultrasound anomalies should be performed to enable further analysis of prospective parents' decision‐making regarding prenatal genetic testing, e.g. preferences, reasons for choosing a certain test modality, informed decision‐making, decisional conflict and decision regret. This study is limited by the fact that, although complete follow‐up of pregnancy outcomes was ascertained, diagnostic genetic testing was not performed when sonography or newborn examination did not call for it.

For pregnancies included in this study, fear of a procedure‐related miscarriage and unwillingness to terminate the pregnancy in case of fetal chromosomal abnormality were often noted as reasons for declining invasive diagnostic testing. Unfortunately, the data extracted from medical records were insufficient for further analysis of prospective parents' decision‐making. Prospective studies comparing the utility of invasive diagnostic testing with the utility of NIPT for genetic evaluation of pregnancies with ultrasound anomalies should be performed to enable further analysis of prospective parents' decision‐making regarding prenatal genetic testing, e.g. preferences, reasons for choosing a certain test modality, informed decision‐making, decisional conflict and decision regret. This study is limited by the fact that, although complete follow‐up of pregnancy outcomes was ascertained, diagnostic genetic testing was not performed when sonography or newborn examination did not call for it. In conclusion, NIPT should not be recommended for genetic evaluation of the etiology of ultrasound anomalies, as both resolution and sensitivity, or negative predictive value, are inferior to those of conventional karyotyping and microarray analysis. Nonetheless, some pregnant women consider NIPT to be an acceptable alternative to invasive diagnostic testing. Supporting information Table S1 Ultrasound anomaly indicating genetic testing, genetic test results and pregnancy outcome in 251 women who had non‐invasive prenatal testing (NIPT)

In conclusion, NIPT should not be recommended for genetic evaluation of the etiology of ultrasound anomalies, as both resolution and sensitivity, or negative predictive value, are inferior to those of conventional karyotyping and microarray analysis. Nonetheless, some pregnant women consider NIPT to be an acceptable alternative to invasive diagnostic testing. Supporting information Table S1 Ultrasound anomaly indicating genetic testing, genetic test results and pregnancy outcome in 251 women who had non‐invasive prenatal testing (NIPT) Table S2 Overview of genetic findings by non‐invasive prenatal testing (NIPT) and diagnostic genetic testing in 21 multiple pregnancies with ultrasound (US) anomaly Click here for additional data file. ACKNOWLEDGMENTS We are grateful to all healthcare professionals who assisted in obtaining complete follow‐up of pregnancy outcomes. This study was supported financially by the Foundation for Prenatal Screening in the Nijmegen Region.

INTRODUCTION Adnexal masses are common. Although only a minority are malignant, their management, and therefore both patient morbidity and mortality, depends on their correct preoperative differentiation. For benign masses, conservative management or laparoscopic and fertility‐sparing surgery is preferred. Laparoscopy is associated with reduced morbidity and lower cost when compared with laparotomy1. In the case of malignancy, however, more extensive surgery is necessary, preferably performed in an oncology center. This is essential to optimize care and thereby survival of the patient2, 3. Ultrasound examination, more specifically subjective assessment by an expert examiner, is considered the best way to differentiate malignant from benign adnexal masses prior to surgery4. However, an expert examiner is not always available.

ter. This is essential to optimize care and thereby survival of the patient2, 3. Ultrasound examination, more specifically subjective assessment by an expert examiner, is considered the best way to differentiate malignant from benign adnexal masses prior to surgery4. However, an expert examiner is not always available. Various ultrasound‐based prediction models and scoring systems have been developed to support the diagnosis of adnexal masses by less experienced examiners. The Risk of Malignancy Index (RMI) is one such scoring system5, 6, 7, and is currently recommended by many national guidelines. However, performance of this model is poor4, 8. Other models, with better test accuracy, include the International Ovarian Tumour Analysis (IOTA) simple ultrasound‐based rules (‘simple rules’)9 and IOTA Logistic Regression model 2 (LR2)10. Another model with excellent test performance was developed recently: the Assessment of Different NEoplasias in the adneXa (ADNEX) model11. This model predicts not only whether a mass is malignant, but also, to a certain extent, the type of malignancy. Insight into the specific tumor type makes it possible to optimize treatment, which may reduce morbidity and enhance the chances of survival3. For example, the distinction between malignancy and borderline malignancy is relevant for the treatment of premenopausal women in the context of fertility preservation.

ignancy. Insight into the specific tumor type makes it possible to optimize treatment, which may reduce morbidity and enhance the chances of survival3. For example, the distinction between malignancy and borderline malignancy is relevant for the treatment of premenopausal women in the context of fertility preservation. The aim of this study was to validate externally the performance of the ADNEX model, and compare it with that of other frequently used models, in the differentiation between benign and malignant adnexal masses.

ignancy. Insight into the specific tumor type makes it possible to optimize treatment, which may reduce morbidity and enhance the chances of survival3. For example, the distinction between malignancy and borderline malignancy is relevant for the treatment of premenopausal women in the context of fertility preservation. The aim of this study was to validate externally the performance of the ADNEX model, and compare it with that of other frequently used models, in the differentiation between benign and malignant adnexal masses. METHODS Study design and setting This was a retrospective, single‐center, diagnostic accuracy study, conducted at a tertiary care hospital using data collected prospectively between July 2011 and July 2015. A single ultrasonographer (T.V.G.) with more than 10 years' experience in gynecological ultrasound (Level‐3 examiner) assessed all consecutively recruited patients with adnexal pathology12. All women underwent transvaginal or transrectal grayscale and color Doppler ultrasound examination, using a Voluson E8 (GE Healthcare Ultrasound, Milwaukee, WI, USA) ultrasound machine. If the mass was too large to be seen entirely by transvaginal ultrasound, or if malignancy was suspected, transabdominal ultrasound was also performed. The operator assessed the sonographic tumor morphology based on the nomenclature of the IOTA Group13, recording the ultrasound findings in a secure electronic data‐collection system (Astraia version 1.23.6, Astraia Software GmbH, Munich, Germany) together with demographic data, tumor markers and tumor diagnosis based on subjective assessment. The complete list of prospectively collected ultrasound features is shown in Table S1. Along with the subjective classification as benign or malignant, the ultrasound examiner noted his level of confidence (certain, probable or uncertain). All assessments were done prior to obtaining the histological diagnosis.

ssment. The complete list of prospectively collected ultrasound features is shown in Table S1. Along with the subjective classification as benign or malignant, the ultrasound examiner noted his level of confidence (certain, probable or uncertain). All assessments were done prior to obtaining the histological diagnosis. Patients were excluded when no pathology result was obtained, when the pathology result was known before the ultrasound examination (from transabdominal biopsy in the case of metastasis), when pathology was obtained > 120 days after the ultrasound examination and when a patient had previously undergone a bilateral oophorectomy. Patients with a previous hysterectomy who were 50 years of age or older and patients with amenorrhea of more than 1 year were defined as postmenopausal. Pathology was the clinical reference standard used for all patients in this study. Results were obtained by either surgery or biopsy of a metastasis and added to the database. The pathologist was unaware of the results of the ultrasound examination. Tumors were classified according to the World Health Organization International Classification of Ovarian Tumors14. Tumor stage was defined according to the International Federation of Gynecologists and Obstetricians (FIGO) 2012 classification15.

The pathologist was unaware of the results of the ultrasound examination. Tumors were classified according to the World Health Organization International Classification of Ovarian Tumors14. Tumor stage was defined according to the International Federation of Gynecologists and Obstetricians (FIGO) 2012 classification15. The study was approved by the Medical Research Ethics Committee of the Maastricht University Medical Center in The Netherlands. According to Dutch law, this study was not subject to formal ethics committee assessment and therefore no informed consent of patients was required. STARD guidelines16 were followed for the conduct, analysis and reporting of the study. Prediction models Risk of malignancy was determined by four prediction models and subjective assessment by the expert ultrasonographer.

The study was approved by the Medical Research Ethics Committee of the Maastricht University Medical Center in The Netherlands. According to Dutch law, this study was not subject to formal ethics committee assessment and therefore no informed consent of patients was required. STARD guidelines16 were followed for the conduct, analysis and reporting of the study. Prediction models Risk of malignancy was determined by four prediction models and subjective assessment by the expert ultrasonographer. The ADNEX model11 includes nine variables: age (years), serum CA 125 level (U/mL), type of center (oncology center/other hospital), maximum diameter of the lesion (mm), proportion of solid tissue (%), number of papillary projections (0/1/2/3/> 3), more than 10 cyst locules (yes/no), acoustic shadow (yes/no) and ascites (yes/no). The formula for the risk calculation can be found in the original article11; for use in clinical practice, an application is available (http://www.iotagroup.org/adnexmodel). The outcome of this model is an absolute risk estimate (expressed as a percentage) for five different types of adnexal pathology: benign, borderline, Stage‐I invasive, Stage‐II–IV invasive and secondary metastatic. Furthermore, a risk estimate for the overall risk of malignancy is given (which is the sum of the estimates for all subtypes of malignancy). A cut‐off of ≥ 10% for the overall risk of malignancy was used to predict malignancy11, 17.

y: benign, borderline, Stage‐I invasive, Stage‐II–IV invasive and secondary metastatic. Furthermore, a risk estimate for the overall risk of malignancy is given (which is the sum of the estimates for all subtypes of malignancy). A cut‐off of ≥ 10% for the overall risk of malignancy was used to predict malignancy11, 17. The IOTA simple rules model (Table 1)9 includes five ultrasound features suggestive of benignity (B‐features) and five features suggestive of malignancy (M‐features). If one or more B‐features are present in the absence of M‐features, the mass is classified as benign, and vice versa. If both B‐ and M‐features exist or if none of the 10 features is present, the simple rules yields an inconclusive result. Two different approaches were used for these difficult‐to‐diagnose masses: use of subjective assessment by the expert ultrasonographer as a second‐stage test, and classification of all inconclusive masses as malignant. Table 1 International Ovarian Tumour Analysis (IOTA) simple ultrasound‐based rules9 for prediction of malignancy in adnexal mass, divided into five benign (B)‐features and five malignant (M)‐features B1 Unilocular tumor M1 Irregular solid tumor

The IOTA simple rules model (Table 1)9 includes five ultrasound features suggestive of benignity (B‐features) and five features suggestive of malignancy (M‐features). If one or more B‐features are present in the absence of M‐features, the mass is classified as benign, and vice versa. If both B‐ and M‐features exist or if none of the 10 features is present, the simple rules yields an inconclusive result. Two different approaches were used for these difficult‐to‐diagnose masses: use of subjective assessment by the expert ultrasonographer as a second‐stage test, and classification of all inconclusive masses as malignant. Table 1 International Ovarian Tumour Analysis (IOTA) simple ultrasound‐based rules9 for prediction of malignancy in adnexal mass, divided into five benign (B)‐features and five malignant (M)‐features B1 Unilocular tumor M1 Irregular solid tumor B2 Solid component with largest diameter < 7 mm M2 Presence of ascites B3 Presence of acoustic shadow M3 ≥ 4 papillary projections B4 Smooth multilocular tumor with largest diameter < 100 mm M4 Irregular multilocular solid tumor with largest diameter ≥ 100 mm B5 No blood flow (color score 1) M5 Very strong blood flow (color score 4) The LR2 model10 uses six variables to estimate the probability of malignancy: (a) age (years); (b) presence of ascites (yes = 1, no = 0); (c) presence of blood flow within a papillary projection (yes = 1, no = 0); (d) maximum diameter of the solid component (mm; capped at 50 mm); (e) irregular internal cyst walls (yes = 1, no = 0); and (f) presence of acoustic shadow (yes = 1, no = 0). The estimated probability of malignancy for an adnexal tumor is calculated by LR2 as: 1/(1 exp(−z)), where z = −5.3718 + 0.0354a + 1.6159b + 1.1768c + 0.0697d + 0.9586e – 2.9486f. A cut‐off of ≥ 0.1 (≥ 10%) was used to predict malignancy.

t walls (yes = 1, no = 0); and (f) presence of acoustic shadow (yes = 1, no = 0). The estimated probability of malignancy for an adnexal tumor is calculated by LR2 as: 1/(1 exp(−z)), where z = −5.3718 + 0.0354a + 1.6159b + 1.1768c + 0.0697d + 0.9586e – 2.9486f. A cut‐off of ≥ 0.1 (≥ 10%) was used to predict malignancy. The RMI scoring system5, 6, 7 combines the ultrasound features of the mass (U), the menopausal status of the patient (M) and the serum CA 125 level (U/mL) into a risk score (U × M × serum CA 125). The ultrasound features are multilocularity, solid areas, bilaterality, ascites and intra‐abdominal metastases. Three principal variants of the RMI were applied (RMI‐I, RMI‐II and RMI‐III), which differed according to the points attributed to the different ultrasound variables and the menopausal status of the patient (Table 2). A total score of ≥ 200 was used as a cut‐off for malignancy. Table 2 Characteristics of three variants of Risk of Malignancy Index (RMI) scoring system for prediction of malignancy in adnexal mass5, 6, 7 RMI variant Ultrasound score (U) Menopausal status (M) Characteristic Score Characteristic Score RMI‐I No features present 0 Premenopausal 1 1 feature present 1 Postmenopausal 3 ≥ 2 features present 3 RMI‐II ≤ 1 feature present 1 Premenopausal 1 ≥ 2 features present 4 Postmenopausal 4 RMI‐III ≤ 1 feature present 1 Premenopausal 1 ≥ 2 features present 3 Postmenopausal 3 Ultrasound score (U) includes five features: multilocular cyst, solid areas, bilateral cysts, ascites and intra‐abdominal metastases.

tures present 3 RMI‐II ≤ 1 feature present 1 Premenopausal 1 ≥ 2 features present 4 Postmenopausal 4 RMI‐III ≤ 1 feature present 1 Premenopausal 1 ≥ 2 features present 3 Postmenopausal 3 Ultrasound score (U) includes five features: multilocular cyst, solid areas, bilateral cysts, ascites and intra‐abdominal metastases. Total scores for U and M are inserted into the following formula to calculate RMI: U × M × serum CA 125.

tures present 3 RMI‐II ≤ 1 feature present 1 Premenopausal 1 ≥ 2 features present 4 Postmenopausal 4 RMI‐III ≤ 1 feature present 1 Premenopausal 1 ≥ 2 features present 3 Postmenopausal 3 Ultrasound score (U) includes five features: multilocular cyst, solid areas, bilateral cysts, ascites and intra‐abdominal metastases. Total scores for U and M are inserted into the following formula to calculate RMI: U × M × serum CA 125. Statistical analysis All data analyses were performed with IBM SPSS statistics v20 (IBM Corp, Los Angeles, CA, USA) and MedCalc v16.1 (MedCalc Software, Mariakerke, Belgium). For statistical purposes, borderline tumors were considered malignant. In women with bilateral tumors, only the tumor with the most complex ultrasound morphology was included in the statistical analysis. If both masses had the same morphology, the mass with the largest size was used. We calculated sensitivity, specificity, positive and negative predictive values (PPV and NPV) and positive and negative likelihood ratios (LR+ and LR–) for the cut‐off points proposed in the original publications for each model. We also performed a subgroup analysis for pre‐ and postmenopausal patients. Multiple imputation (fully conditional specification) was used to deal with missing values of serum CA 12511, 18. Predictive mean matching regression was applied, using variables from our dataset related to the level of CA 125 (i.e. values included in the ADNEX model and others such as pathology results, previous hysterectomy and parity), or the unavailability of this tumor marker (i.e. a binary indicator with value 1 if results from CA 125 were missing and value 0 if they were not).

variables from our dataset related to the level of CA 125 (i.e. values included in the ADNEX model and others such as pathology results, previous hysterectomy and parity), or the unavailability of this tumor marker (i.e. a binary indicator with value 1 if results from CA 125 were missing and value 0 if they were not). Receiver–operating characteristics (ROC) curves were derived for the ADNEX model, subjective assessment, LR2 and RMI, and summarized by calculating the area under the curve (AUC) with 95% CI using exact methods based on the binomial distribution. To calculate the AUC for subjective assessment, six levels of diagnostic confidence were used (certainly benign; probably benign; uncertain, but most likely benign; uncertain, but most likely malignant; probably malignant; certainly malignant). The method described by DeLong et al.19 was used to calculate statistical significance of differences between AUCs. The McNemar test was used to test the statistical significance of differences in sensitivity and specificity between the various models, when an AUC could not be calculated (i.e. the two different variants of the simple rules). P < 0.05 was considered statistically significant for all comparisons.

erences between AUCs. The McNemar test was used to test the statistical significance of differences in sensitivity and specificity between the various models, when an AUC could not be calculated (i.e. the two different variants of the simple rules). P < 0.05 was considered statistically significant for all comparisons. RESULTS Between July 2011 and July 2015 a total of 851 patients visited our hospital to undergo adnexal ultrasound examination by an expert, and pathology results were obtained for 424 of them. The final cohort consisted of 326 consecutive patients who fulfilled our inclusion criteria, involving 128 (39.3%) premenopausal and 198 (60.7%) postmenopausal patients. A detailed overview of patient inclusion is shown in Figure 1. Patient characteristics and data for the ultrasound features used in the different models are shown in Table 3. Figure 1 Flow diagram summarizing inclusion of patients in the study. US, ultrasound examination. UOG-17225-FIG-0001-bTable 3 Descriptive statistics for patient characteristics and ultrasound features according to tumor type in 326 patients with adnexal mass

RESULTS Between July 2011 and July 2015 a total of 851 patients visited our hospital to undergo adnexal ultrasound examination by an expert, and pathology results were obtained for 424 of them. The final cohort consisted of 326 consecutive patients who fulfilled our inclusion criteria, involving 128 (39.3%) premenopausal and 198 (60.7%) postmenopausal patients. A detailed overview of patient inclusion is shown in Figure 1. Patient characteristics and data for the ultrasound features used in the different models are shown in Table 3. Figure 1 Flow diagram summarizing inclusion of patients in the study. US, ultrasound examination. UOG-17225-FIG-0001-bTable 3 Descriptive statistics for patient characteristics and ultrasound features according to tumor type in 326 patients with adnexal mass Variable Benign (n = 211) Borderline (n = 27) Stage I (n = 18) Stage II–IV (n = 56) Metastatic (n = 14) Age (years) 53.2 50.6 63.1 67.7 64.6 (16.1–87.2) (36.9–65.8) (50.3–68.5) (32.3–87.0) (20.0–87.1) CA 125 (U/mL)* 26.0 61.9 109.5 456.0 78.6 [16.5–27.0] [27.5–295.0] [16.8–361.5] [170.8–1175.0] [27.5–260.8] Missing values for CA 125 31 (14.7) 1 (3.7) 0 (0.0) 0 (0.0) 0 (0.0) Menopausal status Premenopausal 97 (46.0) 15 (55.6) 6 (33.3) 7 (12.5) 3 (21.4) Postmenopausal 114 (54.0) 12 (44.4) 12 (66.7) 49 (87.5) 11 (78.6) Patient pregnant 2 (0.9) 1 (3.7) 0 (0.0) 1 (1.8) 0 (0.0) Family history of ovarian cancer 6 (2.8) 1 (3.7) 0 (0.0) 3 (5.3) 0 (0.0) Laterality of tumor Unilateral 169 (80.1) 21 (77.8) 16 (88.9) 35 (62.5) 10 (71.4) Bilateral 42 (19.9) 6 (22.2) 2 (11.1) 21 (37.5) 4 (28.6) Maximum diameter of lesion (mm) 80.0 155.0 122.5 71.5 105.0 [59.0–115.0] [123.0–229.0] [92.0–214.0] [50.3–102.8] [63.0–133.5] Type of tumor Unilocular 65 (30.8) 0 (0.0) 0 (0.0) 0 (0.0) 1 (7.1) Multilocular 71 (33.6) 6 (22.2) 0 (0.0) 0 (0.0) 0 (0.0) Unilocular‐solid 19 (9.0) 5 (19.5) 2 (11.1) 4 (7.1) 0 (0.0) Multilocular‐solid 35 (16.6) 16 (59.3) 11 (61.1) 21 (37.5) 4 (28.6) Solid 20 (9.5) 0 (0.0) 5 (27.8) 31 (55.4) 9 (64.3) Unclassifiable 1 (0.5) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) Solid tissue 1 (0.5) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) Presence of solid tissue 78 (37.0) 21 (77.8) 18 (100.0) 55 (98.2) 14 (100.0) Maximum diameter of solid tissue (mm) 0.0 50.3 52.5 58.0 60.5 [0.0–16.0] [25.0–60.5] [29.8–93.5] [33.5–80.0] [34.5–123.5] Proportion of solid tissue if present (%) 28.9 27.9 55.9 100.0 100.0 [15.7–100.0] [18.4–46.9] [20.9–100.0] [56.1–100.0] [54.6–100.0] Number of locules 0 21 (10.0) 0 (0.0) 5 (27.8) 31 (55.4) 9 (64.3) 1–4 134 (63.5) 12 (44.4) 3 (16.7) 7 (12.5) 3 (21.4) 5–10 29 (13.7) 4 (14.8) 4 (22.2) 13 (23.2) 1 (7.1) > 10 27 (12.8) 11 (40.7) 6 (33.3) 5 (8.9) 1 (7.1) Number of papillary projections 0 167 (79.1) 13 (48.1) 13 (72.2) 46 (82.1) 12 (85.7) 1 22 (10.4) 3 (11.1) 1 (5.6) 5 (8.9) 1 (7.1) 2 8 (3.8) 1 (3.7) 0 (0.0) 1 (1.8) 0 (0.0) 3 1 (0.5) 1 (3.7) 0 (0.0) 1 (1.8) 0 (0.0) > 3 13 (6.2) 9 (33.3) 4 (22.2) 3 (5.4) 1 (7.1) Blood flow in papillary projections: color Doppler score 1 10

mber of papillary projections 0 167 (79.1) 13 (48.1) 13 (72.2) 46 (82.1) 12 (85.7) 1 22 (10.4) 3 (11.1) 1 (5.6) 5 (8.9) 1 (7.1) 2 8 (3.8) 1 (3.7) 0 (0.0) 1 (1.8) 0 (0.0) 3 1 (0.5) 1 (3.7) 0 (0.0) 1 (1.8) 0 (0.0) > 3 13 (6.2) 9 (33.3) 4 (22.2) 3 (5.4) 1 (7.1) Blood flow in papillary projections: color Doppler score 1 10 6 (50.2) 3 (11.1) 1 (5.6) 3 (5.4) 2 (14.3) 2 75 (35.5) 11 (40.7) 3 (16.7) 11 (19.6) 2 (14.3) 3 24 (11.4) 12 (44.4) 11 (61.1) 24 (42.9) 4 (28.6) 4 6 (2.8) 1 (3.7) 3 (16.7) 18 (32.1) 6 (42.9) Irregular cyst wall 78 (37.0) 20 (74.1) 13 (72.2) 51 (91.1) 13 (92.9) Metastases 2 (0.9) 3 (11.1) 1 (5.6) 34 (60.7) — Acoustic shadow 78 (37.0) 4 (14.8) 2 (11.1) 5 (8.9) 1 (7.1) Ascites 13 (6.2) 6 (22.2) 3 (16.7) 35 (62.5) 2 (14.3) Data are given as n (%), median (range) or median [interquartile range]. * Results based on multiple imputation of missing values. The median interval between ultrasound examination and obtaining the pathology results was 21 days. Results were benign for 211 (64.7%) masses and malignant for 115 (35.3%) masses (Table 4). The most common benign pathologies were cystadenoma, endometrioma, mature teratoma, fibroma and cystadenofibroma. Six benign masses consisted of mixed pathology (two or more different histological subtypes) and therefore could not be categorized into a specific subtype. Table 4 Pathology results of 326 adnexal masses

The median interval between ultrasound examination and obtaining the pathology results was 21 days. Results were benign for 211 (64.7%) masses and malignant for 115 (35.3%) masses (Table 4). The most common benign pathologies were cystadenoma, endometrioma, mature teratoma, fibroma and cystadenofibroma. Six benign masses consisted of mixed pathology (two or more different histological subtypes) and therefore could not be categorized into a specific subtype. Table 4 Pathology results of 326 adnexal masses Pathology n (%) Benign 211 (64.7) Cystadenoma  82 (25.2) Endometriotic cyst  39 (12.0) Mature teratoma  29 (8.9) Fibroma  23 (7.1) Cystadenofibroma  15 (4.6) Salpingitis   6 (1.8) Functional cyst   4 (1.2) Parasalpingeal cyst   2 (0.6) Struma ovarii   2 (0.6) Pseudocyst   2 (0.6) Unknown type   1 (0.3) Mixed   6 (1.8) Borderline  27 (8.3) Serous  11 (3.4) Mucinous  13 (4.0) Other   3 (0.9) Malignant  88 (27.0) Epithelial ovarian cancer  70 (21.5) Stage Stage I  14 (20.0) Stage II   8 (11.4) Stage III  27 (38.6) Stage IV  21 (30.0) Differentiation grade Grade 1   8 (11.4) Grade 2  12 (17.1) Grade 3  46 (65.7) Unknown   4 (5.7) Granulosa cell carcinoma   3 (0.9) Yolk‐sac tumor   1 (0.3) Metastatic tumor  10 (3.1) Non‐primary ovarian carcinoma   4 (1.2) The majority (84.3% (97/115)) of malignancies consisted of epithelial ovarian carcinomas. Almost a quarter of all malignant masses were borderline tumors. Furthermore, 14 patients were diagnosed with extraovarian primary tumors; 10 of these were extraovarian tumors (mainly of gastrointestinal or endometrial origin) with metastases to the ovaries, while the others were primary tumors of rectosigmoid or endometrial origin, mimicking a primary tumor of the ovary.

ne tumors. Furthermore, 14 patients were diagnosed with extraovarian primary tumors; 10 of these were extraovarian tumors (mainly of gastrointestinal or endometrial origin) with metastases to the ovaries, while the others were primary tumors of rectosigmoid or endometrial origin, mimicking a primary tumor of the ovary. Validation of ADNEX model The ADNEX model, at a cut‐off ≥ 10%, had a sensitivity of 0.98 (95% CI, 0.93–1.00) and a specificity of 0.62 (95% CI, 0.55–0.68) (Table 5). The AUC for the overall discrimination between benign and malignant tumors was 0.93 (95% CI, 0.89–0.95). AUCs for discrimination between different tumor subgroups ranged between 0.60 and 0.97 (Table 6). The model was particularly able to distinguish benign from Stage‐II–IV tumors, benign from secondary metastatic cancer and borderline from secondary metastatic cancer. In contrast, discrimination between borderline and Stage‐I tumors and between Stage‐II–IV tumors and secondary metastatic cancer was mediocre (Table 6). Table 5 Diagnostic performance indices for subjective assessment (SA) and four prediction models for differentiation between benign and malignant adnexal masses, in whole study population (n = 326) and in premenopausal (n = 128) and postmenopausal (n = 198) subgroups

Validation of ADNEX model The ADNEX model, at a cut‐off ≥ 10%, had a sensitivity of 0.98 (95% CI, 0.93–1.00) and a specificity of 0.62 (95% CI, 0.55–0.68) (Table 5). The AUC for the overall discrimination between benign and malignant tumors was 0.93 (95% CI, 0.89–0.95). AUCs for discrimination between different tumor subgroups ranged between 0.60 and 0.97 (Table 6). The model was particularly able to distinguish benign from Stage‐II–IV tumors, benign from secondary metastatic cancer and borderline from secondary metastatic cancer. In contrast, discrimination between borderline and Stage‐I tumors and between Stage‐II–IV tumors and secondary metastatic cancer was mediocre (Table 6). Table 5 Diagnostic performance indices for subjective assessment (SA) and four prediction models for differentiation between benign and malignant adnexal masses, in whole study population (n = 326) and in premenopausal (n = 128) and postmenopausal (n = 198) subgroups Assessment method Sensitivity Specificity PPV NPV LR+ LR– All patients ADNEX 0.98 0.62 0.58 0.98 2.56 0.03 (0.93–1.00) (0.55–0.68) (0.51–0.65) (0.94–0.99) (2.15–3.04) (0.01–0.11) SA 0.90 0.91 0.83 0.95 9.54 0.11 (0.83–0.95) (0.86–0.94) (0.76–0.90) (0.90–0.97) (6.26–14.54) (0.06–0.19) IOTA‐SR + mal 0.93 0.68 0.61 0.93 2.51 0.11 (0.86–0.97) (0.61–0.70) (0.54–0.69) (0.87–0.97) (2.07–3.06) (0.06–0.22) IOTA‐SR + SA 0.89 0.90 0.83 0.94 8.91 0.13 (0.81–0.94) (0.85–0.94) (0.75–0.89) (0.89–0.96) (5.91–13.44) (0.08–0.21) LR2 0.93 0.79 0.71 0.95 4.46 0.09 (0.86–0.97) (0.73–0.84) (0.63–0.78) (0.91–0.98) (3.41–5.83) (0.04–0.17) RMI‐I 0.71 0.79 0.65 0.83 3.34 0.36 (0.62–0.79) (0.72–0.84) (0.56–0.73) (0.77–0.88) (2.52–4.44) (0.27–0.49) RMI‐II 0.74 0.73 0.60 0.84 2.74 0.36 (0.65–0.81) (0.66–0.79) (0.51–0.68) (0.77–0.89) (2.14–3.50) (0.26–0.49) RMI‐III 0.71 0.81 0.67 0.84 3.76 0.35 (0.62–0.79) (0.75–0.86) (0.58–0.75) (0.78–0.88) (2.78–5.09) (0.26–0.47) Premenopausal ADNEX 1.00 0.71 0.53 1.00 3.46 0.00 (0.86–1.00) (0.61–0.80) (0.39–0.66) (0.93–1.00) (2.53–4.73) (0–NA) SA 0.84 0.96 0.87 0.95 20.30 0.17 (0.66–0.94) (0.89–0.99) (0.68–0.96) (0.88–0.98) (7.70–53.75) (0.08–0.38) IOTA‐SR + mal 0.94 0.76 0.56 0.97 3.95 0.08 (0.77–0.99) (0.66–0.84) (0.41–0.69) (0.89–1.00) (2.73–5.70) (0.02–0.33) IOTA‐SR + SA 0.87 0.96 0.87 0.96 21.12 0.13 (0.69–0.96) (0.89–0.99) (0.69–0.96) (0.89–0.99) (8.01–55.66) (0.05–0.34) LR2 0.83 0.92 0.76 0.95 10.17 0.18 (0.66–0.94) (0.84–0.96) (0.58–0.89) (0.87–0.98) (5.14–20.10) (0.08–0.39) RMI‐I 0.42 0.94 0.68 0.83 6.78 0.62 (0.25–0.61) (0.86–0.97) (0.43–0.86) (0.75–0.90) (2.82–16.32) (0.46–0.84) RMI‐II 0.45 0.92 0.64 0.84 5.48 0.60 (0.28–0.64) (0.84–0.96) (0.41–0.82) (0.75–0.90) (2.54–11.81) (0.43–0.82) RMI‐III 0.39 0.95 0.71 0.83 7.51 0.65 (0.22–0.58) (0.88–0.98) (0.44–0.89) (0.74–0.89) (2.87–19.65) (0.49–0.86) Postmenopausal ADNEX 0.98 0.54 0.61 0.97 2.10 0.04 (0.91–1.00) (0.44–0.63) (0.52–0.69) (0.88–0.99) (1.72–2.56) (0.01–0.18) SA 0.93 0.86 0.83 0.94 6.62 0.08 (0.86–0.97) (0.78–0.92) (0.74–0.90) (0.87–0.98) (4.19–10.46) (0.04–0.18) IOTA‐SR + mal 0.93 0.61 0.64 0.92 2.41 0.12 (0.85–0.97) (0.52–0.70) (0.55–0.72) (0.83–0.97) (1.89–3.06) (0.05–0.25) IOTA‐SR +

0.97 2.10 0.04 (0.91–1.00) (0.44–0.63) (0.52–0.69) (0.88–0.99) (1.72–2.56) (0.01–0.18) SA 0.93 0.86 0.83 0.94 6.62 0.08 (0.86–0.97) (0.78–0.92) (0.74–0.90) (0.87–0.98) (4.19–10.46) (0.04–0.18) IOTA‐SR + mal 0.93 0.61 0.64 0.92 2.41 0.12 (0.85–0.97) (0.52–0.70) (0.55–0.72) (0.83–0.97) (1.89–3.06) (0.05–0.25) IOTA‐SR +  SA 0.89 0.85 0.82 0.92 5.99 0.13 (0.80–0.95) (0.77–0.91) (0.72–0.89) (0.84–0.96) (3.84–9.34) (0.07–0.23) LR2 0.96 0.68 0.69 0.96 3.05 0.05 (0.89–0.99) (0.59–0.77) (0.60–0.77) (0.89–0.99) (2.32–4.01) (0.02–0.16) RMI‐I 0.82 0.66 0.64 0.83 2.40 0.27 (0.72–0.89) (0.56–0.74) (0.54–0.73) (0.74–0.90) (1.83–3.16) (0.17–0.43) RMI‐II 0.85 0.57 0.59 0.83 1.97 0.27 (0.75–0.91) (0.47–0.66) (0.50–0.68) (0.73–0.90) (1.56–2.48) (0.16–0.45) RMI‐III 0.83 0.69 0.67 0.85 2.71 0.24 (0.73–0.90) (0.60–0.77) (0.57–0.75) (0.76–0.91) (2.03–3.63) (0.15–0.39) Values in parentheses are 95% CI. Prediction models: Assessment of Different NEoplasias in the adneXa (ADNEX) model11; subjective assessment (SA); International Ovarian Tumour Analysis simple ultrasound‐based rules9 (IOTA‐SR), applied both with inconclusive results being considered to be malignant (IOTA‐SR + mal) and with inconclusive results diagnosed by subjective assessment (IOTA‐SR + SA); IOTA Logistic Regression model 210 (LR2); and three variants of Risk of Malignancy Index (RMI)5, 6, 7. For both ADNEX and LR2 models, cut‐off value of ≥ 0.1 (≥ 10%) was used; for variants of RMI model, cut‐off value of ≥ 200 was used.

Prediction models: Assessment of Different NEoplasias in the adneXa (ADNEX) model11; subjective assessment (SA); International Ovarian Tumour Analysis simple ultrasound‐based rules9 (IOTA‐SR), applied both with inconclusive results being considered to be malignant (IOTA‐SR + mal) and with inconclusive results diagnosed by subjective assessment (IOTA‐SR + SA); IOTA Logistic Regression model 210 (LR2); and three variants of Risk of Malignancy Index (RMI)5, 6, 7. For both ADNEX and LR2 models, cut‐off value of ≥ 0.1 (≥ 10%) was used; for variants of RMI model, cut‐off value of ≥ 200 was used. LR–, negative likelihood ratio; LR+, positive likelihood ratio; NPV, negative predictive value; PPV, positive predictive value. Table 6 Performance of Assessment of Different NEoplasias in the adneXa (ADNEX) model11 for five tumor types, expressed as area under the receiver–operating characteristics curve (AUC) Tumor type AUC (95% CI) Benign vs borderline 0.81 (0.75–0.86) Benign vs Stage I 0.87 (0.84–0.91) Benign vs Stage II–IV 0.97 (0.94–0.99) Benign vs metastatic 0.93 (0.89–0.96) Borderline vs Stage I 0.60 (0.44–0.74) Borderline vs Stage II–IV 0.87 (0.78–0.93) Borderline vs metastatic 0.90 (0.77–0.97) Stage I vs Stage II–IV 0.82 (0.71–0.90) Stage I vs metastatic 0.72 (0.53–0.86) Stage II–IV vs metastatic 0.67 (0.55–0.78)

nign vs Stage II–IV 0.97 (0.94–0.99) Benign vs metastatic 0.93 (0.89–0.96) Borderline vs Stage I 0.60 (0.44–0.74) Borderline vs Stage II–IV 0.87 (0.78–0.93) Borderline vs metastatic 0.90 (0.77–0.97) Stage I vs Stage II–IV 0.82 (0.71–0.90) Stage I vs metastatic 0.72 (0.53–0.86) Stage II–IV vs metastatic 0.67 (0.55–0.78) ADNEX model vs other methods When comparing overall test performance, expressed as AUC, subjective assessment performed significantly better than did the ADNEX model (P = 0.01), with AUCs of 0.96 (95% CI, 0.93–0.98) and 0.93 (95% CI, 0.89–0.95), respectively (Table 7 and Figure 2). The difference between the ADNEX model and LR2 (AUC, 0.92 (95% CI, 0.89–0.95)) was not significant (P = 0.60). The AUCs of all variants of the RMI were significantly lower than those of the other methods in our comparison (all P < 0.001). Table 7 Pairwise receiver–operating characteristics (ROC) curve comparisons expressed as differences in area under the curve (AUC) and P‐values calculated for whole study population SA LR2 RMI‐I RMI‐II RMI‐III ADNEX 0.027 (0.008–0.047)* 0.005 (−0.015–0.025)† 0.075 (0.040–0.109)† 0.108 (0.066–0.149)† 0.088 (0.049–0.127)† P = 0.01 P = 0.5968 P < 0.0001 P < 0.0001 P < 0.0001 SA — 0.033 (0.007–0.058)† 0.102 (0.062–0.141)† 0.135 (0.089–0.182)† 0.115 (0.072–0.159)† P = 0.0119 P < 0.0001 P < 0.0001 P < 0.0001 LR2 — — 0.069 (0.029–0.110)† 0.103 (0.057–0.148)† 0.082 (0.039–0.126)† P = 0.0009 P < 0.0001 P = 0.0002 RMI‐I — — — 0.033 (0.016–0.051)† 0.013 (0.004–0.022)† P = 0.0003 P = 0.0041 RMI‐II — — — — 0.020 (0.004–0.036)†

P = 0.01 P = 0.5968 P < 0.0001 P < 0.0001 P < 0.0001 SA — 0.033 (0.007–0.058)† 0.102 (0.062–0.141)† 0.135 (0.089–0.182)† 0.115 (0.072–0.159)† P = 0.0119 P < 0.0001 P < 0.0001 P < 0.0001 LR2 — — 0.069 (0.029–0.110)† 0.103 (0.057–0.148)† 0.082 (0.039–0.126)† P = 0.0009 P < 0.0001 P = 0.0002 RMI‐I — — — 0.033 (0.016–0.051)† 0.013 (0.004–0.022)† P = 0.0003 P = 0.0041 RMI‐II — — — — 0.020 (0.004–0.036)† P = 0.0123 RMI‐III — — — — — Prediction models: Assessment of Different NEoplasias in the adneXa (ADNEX) model11; subjective assessment (SA); IOTA Logistic Regression model 210 (LR2); and three variants of Risk of Malignancy Index (RMI)5, 6, 7. Methods in left column are used as reference standard for comparisons: * model in upper row outperforms corresponding model in left column; † model in left column outperforms corresponding model in upper row. Values in parentheses are 95% CI. Figure 2 Receiver–operating characteristics curves for detection of malignant disease (including borderline ovarian tumors) for the Assessment of Different NEoplasias in the adneXa (ADNEX) model11, subjective assessment (SA), International Ovarian Tumour Analysis (IOTA) Logistic Regression model 210 (LR2) and three variants of the Risk of Malignancy Index5, 6, 7 (RMI) in the whole population (n = 326) (a) and in premenopausal (n = 128) (b) and postmenopausal (n = 198) (c) subgroups. AUC, area under the curve.

11, subjective assessment (SA), International Ovarian Tumour Analysis (IOTA) Logistic Regression model 210 (LR2) and three variants of the Risk of Malignancy Index5, 6, 7 (RMI) in the whole population (n = 326) (a) and in premenopausal (n = 128) (b) and postmenopausal (n = 198) (c) subgroups. AUC, area under the curve. UOG-17225-FIG-0002-cFor the study population as a whole, among all the methods assessed, the sensitivity of the ADNEX model (at cut‐off ≥ 10%) was highest, although the specificity was lowest (Table 5). The sensitivity and specificity of subjective assessment differed significantly from those of the ADNEX model (P = 0.01 and P < 0.0001, respectively). The sensitivity and specificity of the simple rules, using subjective assessment in case of inconclusive test results, were comparable to those of subjective assessment and, as for subjective assessment, differed significantly from those of the ADNEX model (P = 0.03 and P < 0.001 for sensitivity and specificity, respectively). When all masses yielding inconclusive results using the simple rules were classified as malignant instead of using subjective assessment, the specificity dropped significantly (P < 0.0001), while the sensitivity remained high (P = 0.06). The three different variants of the RMI (cut‐off ≥ 200) performed worst of all the methods, with sensitivities as low as 0.71 for both RMI‐I and RMI‐III (95% CI, 0.62–0.79 for both), resulting in the largest differences in sensitivity from that of the ADNEX model (P < 0.0001 for both).

itivity remained high (P = 0.06). The three different variants of the RMI (cut‐off ≥ 200) performed worst of all the methods, with sensitivities as low as 0.71 for both RMI‐I and RMI‐III (95% CI, 0.62–0.79 for both), resulting in the largest differences in sensitivity from that of the ADNEX model (P < 0.0001 for both). Optimal cut‐off We calculated optimal cut‐off values for all models with a cut‐off (i.e. ADNEX model, LR2, RMI‐I, RMI‐II and RMI‐III) at a fixed sensitivity of 90% (Table S2). The optimal cut‐off values for both the ADNEX model and LR2 were higher in our population than the values applied in the original articles: ≥ 26.1% for the ADNEX model, with a specificity of 0.76 (95% CI, 0.66–0.85) and ≥ 16.5% for LR2, with a specificity of 0.82 (95% CI, 0.68–0.89). Optimal cut‐off values for RMI‐I, RMI‐II and RMI‐III, on the other hand, were lower in our population (≥ 63.7%, ≥ 51.3% and ≥ 64.3%, respectively). Pre‐ and postmenopausal subgroups Malignant masses occurred more frequently in postmenopausal than in premenopausal women (42.4% and 24.2%, respectively). Subjective assessment had the highest diagnostic accuracy for differentiating between benign and malignant adnexal masses in both pre‐ and postmenopausal subgroups (Table 5 and Figure 2). Nonetheless, the differences between the AUC for subjective assessment and that for the ADNEX model and for LR2 were not significant (P = 0.65 and P = 0.08, respectively) for premenopausal women (Table S3), while in the postmenopausal subgroup this difference for the ADNEX model was significant (P = 0.02) (Table S4).

. Nonetheless, the differences between the AUC for subjective assessment and that for the ADNEX model and for LR2 were not significant (P = 0.65 and P = 0.08, respectively) for premenopausal women (Table S3), while in the postmenopausal subgroup this difference for the ADNEX model was significant (P = 0.02) (Table S4). DISCUSSION We have shown that the ADNEX model has good overall performance in the differentiation between benign and malignant adnexal masses, with an AUC of 0.93 (95% CI, 0.89–0.95). At the recommended cut‐off of ≥ 10%, the model had high sensitivity; however, this was at the expense of specificity. In our population, the optimal cut‐off of ≥ 26.1% gave somewhat more balanced results for sensitivity and specificity. The model is particularly good at differentiating benign from Stage‐II–IV or secondary metastatic tumors and borderline from secondary metastatic cancer. However, other tumor types could be distinguished less easily. Furthermore, our study suggests that subjective assessment remains superior to the ADNEX model.

ficity. The model is particularly good at differentiating benign from Stage‐II–IV or secondary metastatic tumors and borderline from secondary metastatic cancer. However, other tumor types could be distinguished less easily. Furthermore, our study suggests that subjective assessment remains superior to the ADNEX model. In the original article11, validation AUCs for the ADNEX model were slightly higher than ours, especially for differentiating between various types of malignancy. The model showed better discrimination of borderline from Stage‐I tumors (AUC, 0.75 in the original vs 0.60 in the current study) and Stage‐II–IV from metastatic tumors (AUC, 0.82 in the original vs 0.67 in the current study). This could be due to the distribution of tumor types in each dataset. For example, in the present study, the number of borderline tumors amounted to almost a quarter (23.4%) of all malignancies, and borderline tumors are generally known to be difficult to diagnose20. This also resulted in a slightly lower‐than‐expected test accuracy of subjective assessment. Furthermore, the number of inconclusive results when applying the IOTA simple rules was higher than usual (26.3% in the present study vs 19% in a recent review4). However, general malignancy rate in our study (27%) was similar to that in the original publication (33%). Moreover, this study was conducted in an oncology center, while the original study was performed in both second‐ and third‐level hospitals. Although type of center is the weakest predictor in this model, this could mean that results from our study might not be generalizable11.

to that in the original publication (33%). Moreover, this study was conducted in an oncology center, while the original study was performed in both second‐ and third‐level hospitals. Although type of center is the weakest predictor in this model, this could mean that results from our study might not be generalizable11. The poorest performance was seen for RMI, yet this method is advocated by many guidelines. Although a sensitivity of ≥ 90% is generally considered most important in the preoperative diagnosis of ovarian carcinoma, the sensitivity of RMI‐III was only 71% in our study. This is in accordance with the sensitivity of 0.71 (95% CI, 0.67–0.75) for RMI‐III reported in a recent review of 18 studies validating RMI4. Thus, more than a quarter of ovarian carcinomas will be missed, leading to incorrect treatment of these masses and subsequently to deterioration of the prognosis of these patients3, 21.

in accordance with the sensitivity of 0.71 (95% CI, 0.67–0.75) for RMI‐III reported in a recent review of 18 studies validating RMI4. Thus, more than a quarter of ovarian carcinomas will be missed, leading to incorrect treatment of these masses and subsequently to deterioration of the prognosis of these patients3, 21. This external validation study compared the ADNEX model with other frequently used models to evaluate its added value. It is a strength of our study that the collection of clinical and ultrasound data was meticulous and prospective, in accordance with IOTA nomenclature and measurement techniques, based on real‐time ultrasound and with blinding to pathology results. Although data analysis was done retrospectively, this was not regarded a limitation since previous research on the effects of design‐related biases in studies assessing diagnostic tests showed that a retrospective design is not associated with overestimation or underestimation of diagnostic accuracy22. Overestimation may occur in diagnostic accuracy studies that use different reference tests or with inadequate blinding; this was not the case in our study.

d biases in studies assessing diagnostic tests showed that a retrospective design is not associated with overestimation or underestimation of diagnostic accuracy22. Overestimation may occur in diagnostic accuracy studies that use different reference tests or with inadequate blinding; this was not the case in our study. A limitation of this study is that CA 125 values were missing in 32 (9.8%) patients. When an adnexal mass gives the impression of being completely benign from the overall clinical picture and morphology on ultrasound, clinicians may be less inclined to determine the CA 125 level preoperatively. Had we excluded these patients, we would have introduced selection bias, since all but one of the cases in which CA 125 data were missing were benign (the exception concerned a mucinous borderline tumor). Another potential limitation is that we included both pregnant patients (n = 4) and non‐primary ovarian carcinomas (n = 4). The level of CA 125 can rise during pregnancy, which could lead to an overestimation of the risk of malignancy by models incorporating CA 12523. However, an analysis performed on all patients except the pregnant patients confirmed that pregnancy hardly influenced the results of the ADNEX model (data not shown). The non‐primary ovarian carcinomas were included only if the ultrasound findings were suspicious for ovarian pathology. This is in accordance with daily clinical practice, because the risk of malignancy is estimated after ultrasound and before surgery, and therefore before pathology results revealing non‐primary ovarian carcinoma become available. Finally, ultrasound examinations in our study were performed by an expert ultrasonographer. It remains to be shown if the ADNEX model retains its performance when applied by non‐experts.

after ultrasound and before surgery, and therefore before pathology results revealing non‐primary ovarian carcinoma become available. Finally, ultrasound examinations in our study were performed by an expert ultrasonographer. It remains to be shown if the ADNEX model retains its performance when applied by non‐experts. Using the ADNEX model, absolute risk estimates for benign tumors and four types of malignancy can be obtained with acceptable diagnostic performance. However, how to use the model clinically is not straightforward, as also observed by Van Calster et al.17. Two options are available. First, a cut‐off can be used, such as the one applied in this study. However, rigid use of a cut‐off may result in suboptimal and even unethical judgment, according to Van Calster et al.17. Furthermore, this can only be used to distinguish between benign and malignant masses, thereby losing the advantage of a polytomous model (i.e. a model differentiating between more than two subgroups). Second, an assessment per tumor type can be made to estimate how the predicted risk per type relates to the baseline risk. This requires certain calculations (not supplied by the IOTA application for the ADNEX model) and can also be difficult to interpret.

(i.e. a model differentiating between more than two subgroups). Second, an assessment per tumor type can be made to estimate how the predicted risk per type relates to the baseline risk. This requires certain calculations (not supplied by the IOTA application for the ADNEX model) and can also be difficult to interpret. In this study the ADNEX model was used as a single test, but it can also be applied as a two‐step triage test. For example, when results of the ADNEX model are between certain values (e.g. 5–25%), subjective assessment can be used as a second‐line test to increase diagnostic accuracy. The same kind of triage test could be performed with the other models investigated. In conclusion, the ADNEX model can be used as a good alternative to subjective assessment in the estimation of risk of malignancy of adnexal masses. However, the advantage of the ADNEX model as a polytomous model for the differentiation between various subtypes of malignancy was modest in our study. More guidance on how to use the ADNEX model in clinical practice would be useful. Supporting information Table S1 List of prospectively collected ultrasound features (per adnexal mass) Table S2 Optimal cut‐off and corresponding performance indices for models with a cut‐off value at a fixed sensitivity of 90% Tables S3 and S4 Pairwise receiver–operating characteristics (ROC) curve comparisons expressed as differences in the area under the curve (AUC) and P‐values calculated for premenopausal (Table S3) and postmenopausal (Table S4) patients Click here for additional data file.

Table S2 Optimal cut‐off and corresponding performance indices for models with a cut‐off value at a fixed sensitivity of 90% Tables S3 and S4 Pairwise receiver–operating characteristics (ROC) curve comparisons expressed as differences in the area under the curve (AUC) and P‐values calculated for premenopausal (Table S3) and postmenopausal (Table S4) patients Click here for additional data file. ACKNOWLEDGMENTS This study received funding from the Academic Fund, Maastricht University Medical Center+, The Netherlands and the CZ Fund, The Netherlands. We thank Ben van Calster from the Department of Development and Regeneration, KU Leuven, Belgium for his help with the multiple imputation analysis.

INTRODUCTION Pre‐eclampsia (PE) is a pregnancy‐related complication that affects 3–5% of all pregnancies1 and is a leading cause of maternal and perinatal morbidity and mortality worldwide2. Normal pregnancy is a state of mild systemic inflammation3, 4, whereas PE is associated with exaggerated inflammation5, 6. Soluble fms‐like tyrosine kinase‐1 (sFlt‐1) is an antiangiogenic factor and placental growth factor (PlGF) is a proangiogenic factor produced by the placenta7, 8. In cases of PE, serum concentrations of sFlt‐1 are increased9, whereas those of PlGF are decreased10. An imbalance between sFlt‐1 and PlGF11, 12, 13, together with exaggerated inflammation6, play a major role in the development of endothelial dysfunction that leads to the development of PE. Antiangiogenesis also contributes to the development of cardiovascular disease (CVD). In the last few years several studies have shown that sFlt‐1 levels are higher in individuals with acute myocardial infarction than in those without14, 15, 16. Circulating sFlt‐1 is an effective biomarker for predicting the progression of heart failure in subjects with CVD14, 15. Endothelial dysfunction is the key factor in the pathogenesis of atherosclerosis and CVD17, and meta‐analyses have shown that PE is an independent risk factor for subsequent CVD18, 19.

out14, 15, 16. Circulating sFlt‐1 is an effective biomarker for predicting the progression of heart failure in subjects with CVD14, 15. Endothelial dysfunction is the key factor in the pathogenesis of atherosclerosis and CVD17, and meta‐analyses have shown that PE is an independent risk factor for subsequent CVD18, 19. According to histomorphometry20 and intravascular high‐frequency ultrasonography14, 21, aging and the development of atherosclerosis are associated with increased arterial intima thickness and decreased media thickness. However, these differential changes are not observed by means of conventional measurement of common carotid artery (CCA) intima–media thickness (IMT). Therefore, our group has used high‐frequency ultrasonography to assess intima and media thicknesses separately, in order to calculate the intima to media (I/M) ratio. Using this method, we have shown that women with PE have more vascular damage (preclinical atherosclerosis) than those with normal pregnancy, at the time of PE diagnosis22, 1 year postpartum22 and about 10 years later23. In contrast, conventional CCA‐IMT measurement is unable to reveal any cardiovascular risk at any of these time points22, 23. The aims of this study were to investigate whether higher serum levels of sFlt‐1 and an elevated sFlt‐1/PlGF ratio in women with PE reflect the degree of preclinical atherosclerosis, as estimated by high‐frequency ultrasonography, during pregnancy and at 1 year postpartum.

According to histomorphometry20 and intravascular high‐frequency ultrasonography14, 21, aging and the development of atherosclerosis are associated with increased arterial intima thickness and decreased media thickness. However, these differential changes are not observed by means of conventional measurement of common carotid artery (CCA) intima–media thickness (IMT). Therefore, our group has used high‐frequency ultrasonography to assess intima and media thicknesses separately, in order to calculate the intima to media (I/M) ratio. Using this method, we have shown that women with PE have more vascular damage (preclinical atherosclerosis) than those with normal pregnancy, at the time of PE diagnosis22, 1 year postpartum22 and about 10 years later23. In contrast, conventional CCA‐IMT measurement is unable to reveal any cardiovascular risk at any of these time points22, 23. The aims of this study were to investigate whether higher serum levels of sFlt‐1 and an elevated sFlt‐1/PlGF ratio in women with PE reflect the degree of preclinical atherosclerosis, as estimated by high‐frequency ultrasonography, during pregnancy and at 1 year postpartum. SUBJECTS AND METHODS Women diagnosed with PE and women with normal pregnancy and pregnancy outcomes were recruited in 2007–2010. The method of recruitment of this population has been described extensively in our previous study22. The local ethics committee of the Medical Faculty of Uppsala University approved the study protocol and informed written consent was obtained from each woman included in the study.

outcomes were recruited in 2007–2010. The method of recruitment of this population has been described extensively in our previous study22. The local ethics committee of the Medical Faculty of Uppsala University approved the study protocol and informed written consent was obtained from each woman included in the study. PE was defined as new‐onset hypertension (systolic blood pressure (SBP) ≥ 140 mmHg and/or diastolic blood pressure (DBP) ≥ 90 mmHg observed on at least two occasions ≥ 6 h apart) combined with proteinuria (≥ 2 on a dipstick test or a 24‐h urine sample showing leakage of ≥ 300 mg albumin/24 h) after 20 weeks' gestation. PE was diagnosed as early onset if it occurred before 34 weeks' gestation and late onset if it occurred at or after 34 weeks' gestation. The condition was classified as severe when the increase in blood pressure was marked (SBP ≥ 160 mmHg and/or DBP ≥ 110 mmHg) and/or proteinuria was excessive (≥ 5000 mg/24 h). Among women in the normal pregnancy group, mean gestational age at inclusion was similar to that in the PE group. Normal pregnancy was defined as a normotensive pregnancy resulting in term delivery (≥ 37 weeks) of an appropriate‐weight infant (within ± 2 SD of the mean birth weight for gestational age)24.

PE was defined as new‐onset hypertension (systolic blood pressure (SBP) ≥ 140 mmHg and/or diastolic blood pressure (DBP) ≥ 90 mmHg observed on at least two occasions ≥ 6 h apart) combined with proteinuria (≥ 2 on a dipstick test or a 24‐h urine sample showing leakage of ≥ 300 mg albumin/24 h) after 20 weeks' gestation. PE was diagnosed as early onset if it occurred before 34 weeks' gestation and late onset if it occurred at or after 34 weeks' gestation. The condition was classified as severe when the increase in blood pressure was marked (SBP ≥ 160 mmHg and/or DBP ≥ 110 mmHg) and/or proteinuria was excessive (≥ 5000 mg/24 h). Among women in the normal pregnancy group, mean gestational age at inclusion was similar to that in the PE group. Normal pregnancy was defined as a normotensive pregnancy resulting in term delivery (≥ 37 weeks) of an appropriate‐weight infant (within ± 2 SD of the mean birth weight for gestational age)24. The women were examined first during pregnancy and thereafter at about 1 year after delivery (postpartum). At the postpartum examination, all but three of the women with PE had restarted menstruation and all but three had stopped breastfeeding. Among the women with normal pregnancy, all but two had restarted menstruation and all had stopped breastfeeding. The women who had not restarted menstruation were taking contraceptive medication and the women who were still breastfeeding did so partially and had restarted menstruation.

three had stopped breastfeeding. Among the women with normal pregnancy, all but two had restarted menstruation and all had stopped breastfeeding. The women who had not restarted menstruation were taking contraceptive medication and the women who were still breastfeeding did so partially and had restarted menstruation. Assessment during pregnancy and postpartum Based on routine early second‐trimester ultrasonographic dating, gestational age was defined in terms of completed weeks. At inclusion, data on maternal age, reproductive history, smoking habits and height were collected. Maternal weight, enabling calculation of body mass index (BMI), and blood pressure were monitored at both prenatal and postpartum visits. Blood pressure was measured manually in women with PE and automatically in controls, in the upper right arm after about 15 min rest, with the woman in a supine position, using Umedico (Helsinborg, Sweden) blood pressure equipment (cuff size 12 × 35 cm or a size appropriate for the arm circumference). Mean arterial pressure (MAP), which is a better predictor of PE than are SBP and DBP, was calculated as DBP + (SBP – DBP)/325. Data were collected from the delivery records with regard to possible pregnancy‐related complications, gestational age at delivery, mode of delivery and birth weight of the infant. Small‐for‐gestational age (SGA) and large‐for‐gestational age were defined as infants with a birth weight > 2 SD below or above, respectively, the reference population's mean birth weight for gestational age24.

regnancy‐related complications, gestational age at delivery, mode of delivery and birth weight of the infant. Small‐for‐gestational age (SGA) and large‐for‐gestational age were defined as infants with a birth weight > 2 SD below or above, respectively, the reference population's mean birth weight for gestational age24. A venous blood sample was collected from each woman at both examinations. The samples were kept at room temperature (22°C) for about 30 min before being centrifuged for 10 min at 2000 g. Serum samples were separated and stored at −70 °C until required for analysis of the levels of sFlt‐1 and PlGF.

regnancy‐related complications, gestational age at delivery, mode of delivery and birth weight of the infant. Small‐for‐gestational age (SGA) and large‐for‐gestational age were defined as infants with a birth weight > 2 SD below or above, respectively, the reference population's mean birth weight for gestational age24. A venous blood sample was collected from each woman at both examinations. The samples were kept at room temperature (22°C) for about 30 min before being centrifuged for 10 min at 2000 g. Serum samples were separated and stored at −70 °C until required for analysis of the levels of sFlt‐1 and PlGF. Enzyme‐linked immunosorbent assays Levels of sFlt‐1 and PlGF were analyzed using commercially available enzyme‐linked immunosorbent assay (ELISA) kits. The ELISAs were performed without knowledge of the clinical diagnosis and the kits (R&D Systems, Minneapolis, MN, USA) contained microtiter plates on which specific monoclonal antibodies were coated. Standards and samples were pipetted into the wells and the peptide was bound to the immobilized antibodies. After washing, an enzyme‐conjugated polyclonal antipeptide antibody was added to the wells. After incubation and washing, a substrate solution was added. Development was stopped and absorbance was measured using SpectraMax 250 equipment (Molecular Devices, Sunnyvale, CA, USA). The peptide concentrations in the samples were determined by comparing the optical density of the sample against the standard curve. The manufacturer determined the specificity of the assays, which do not exhibit any cross‐reactivity with a panel of other recombinant human and mouse cytokines. The detection limit of the PlGF test was 10 pg/mL and PlGF levels below this limit were assigned as 10 pg/mL.

ensity of the sample against the standard curve. The manufacturer determined the specificity of the assays, which do not exhibit any cross‐reactivity with a panel of other recombinant human and mouse cytokines. The detection limit of the PlGF test was 10 pg/mL and PlGF levels below this limit were assigned as 10 pg/mL. High‐frequency ultrasonography of arterial wall The left CCA wall layers were imaged (Figure  1) using high‐resolution ultrasonographic equipment fitted with a broadband probe with 22‐MHz center frequency (Collagenoson®, Minhorst Company, Meudt, Germany). The method has been described extensively elsewhere26, 27. Point estimates of the arterial wall, not adjusted to the cardiac cycle, were obtained and about 20 point estimates were saved on a computer by one researcher (M.L.). Individual arterial wall layer dimensions were measured offline for all participants by another researcher (T.A.) who was blinded to the study group and time of assessment. The means of about 10 technically acceptable measurements were calculated and used in the analysis. In our laboratory, the coefficient of variation was 3.9% for intima thickness and 3.4% for media thickness26. Figure 1 Layers of common carotid arterial wall, examined by non‐invasive 22‐MHz ultrasonography. A, adventitia; C, cutis; I, intima; M, media; SC, subcutis. Image reused with permission from the American Heart Association22.

High‐frequency ultrasonography of arterial wall The left CCA wall layers were imaged (Figure  1) using high‐resolution ultrasonographic equipment fitted with a broadband probe with 22‐MHz center frequency (Collagenoson®, Minhorst Company, Meudt, Germany). The method has been described extensively elsewhere26, 27. Point estimates of the arterial wall, not adjusted to the cardiac cycle, were obtained and about 20 point estimates were saved on a computer by one researcher (M.L.). Individual arterial wall layer dimensions were measured offline for all participants by another researcher (T.A.) who was blinded to the study group and time of assessment. The means of about 10 technically acceptable measurements were calculated and used in the analysis. In our laboratory, the coefficient of variation was 3.9% for intima thickness and 3.4% for media thickness26. Figure 1 Layers of common carotid arterial wall, examined by non‐invasive 22‐MHz ultrasonography. A, adventitia; C, cutis; I, intima; M, media; SC, subcutis. Image reused with permission from the American Heart Association22. UOG-15981-FIG-0001-cStatistical analysis Median and interquartile range were used to present the data. Differences in distributions were tested using the chi‐square test. Between‐group differences in continuous variables were tested by the Mann–Whitney U‐test and within‐group differences by Wilcoxon's signed‐rank test. Spearman's rank correlation test was used to assess correlations between serum levels of sFlt‐1 and PlGF, and the sFlt‐1/PlGF ratio vs arterial wall layer dimensions and cardiovascular risk factors in the combined groups (PE and normal pregnancy), justified by substantial overlapping between groups with regard to sFlt‐1 and PlGF levels, and sFlt‐1/PlGF ratios and similar directions in the associations (Figure  2). Multivariate linear regression analysis was used to assess if the differences in angiogenic factors and arterial wall layer dimensions between the groups remained significant after adjustment for possible confounders. The level of significance was set at P < 0.05. Statistical analyses were performed using SPSS software for Windows (PASW statistics, version 20.0, IBM Corp., Armonk, NY, USA).

nces in angiogenic factors and arterial wall layer dimensions between the groups remained significant after adjustment for possible confounders. The level of significance was set at P < 0.05. Statistical analyses were performed using SPSS software for Windows (PASW statistics, version 20.0, IBM Corp., Armonk, NY, USA). Figure 2 Correlation between soluble fms‐like tyrosine kinase‐1 (sFlt‐1) and intima thickness (a), intima:media ratio (b) and intima–media thickness (IMT) (c) of common carotid artery in women with pre‐eclampsia () and those with normal pregnancy () at study inclusion. (a) rs = 0.51, P < 0.0001; (b) rs = 0.50, P < 0.0001; (c) rs = −0.11, P = 0.22.

soluble fms‐like tyrosine kinase‐1 (sFlt‐1) and intima thickness (a), intima:media ratio (b) and intima–media thickness (IMT) (c) of common carotid artery in women with pre‐eclampsia () and those with normal pregnancy () at study inclusion. (a) rs = 0.51, P < 0.0001; (b) rs = 0.50, P < 0.0001; (c) rs = −0.11, P = 0.22. UOG-15981-FIG-0002-bRESULTS Fifty‐five women with PE and 64 with normal pregnancy were recruited to the study. At the postpartum examination, five women in the PE group were pregnant again and two did not wish to participate. Among the women with normal pregnancy, four were pregnant again, one did not wish to participate and one had moved away from Sweden. Thus, 48 women in the PE group and 58 in the normal pregnancy group were included in the postpartum evaluation. Demographic data of the study population are shown in Table  1 and have been described in our previous publication22. Of the women with PE, 42% had early‐onset PE, 69% had severe PE and 86% were on antihypertensive medication at the time of inclusion. Gestational age at delivery was on average 3 weeks earlier in the PE group than in the normal pregnancy group (P < 0.001). Infants born to mothers with PE had lower birth weights than those born to mothers with normal pregnancy, even after adjustment for gestational age. Table 1 Characteristics of 55 women with pre‐eclampsia (PE) and 64 women with normal pregnancy

UOG-15981-FIG-0002-bRESULTS Fifty‐five women with PE and 64 with normal pregnancy were recruited to the study. At the postpartum examination, five women in the PE group were pregnant again and two did not wish to participate. Among the women with normal pregnancy, four were pregnant again, one did not wish to participate and one had moved away from Sweden. Thus, 48 women in the PE group and 58 in the normal pregnancy group were included in the postpartum evaluation. Demographic data of the study population are shown in Table  1 and have been described in our previous publication22. Of the women with PE, 42% had early‐onset PE, 69% had severe PE and 86% were on antihypertensive medication at the time of inclusion. Gestational age at delivery was on average 3 weeks earlier in the PE group than in the normal pregnancy group (P < 0.001). Infants born to mothers with PE had lower birth weights than those born to mothers with normal pregnancy, even after adjustment for gestational age. Table 1 Characteristics of 55 women with pre‐eclampsia (PE) and 64 women with normal pregnancy Characteristic PE Normal Maternal age (years) 30 (26–34) 30 (28–33) GA at examination (weeks) 35 (27–37) 36 (34–37) Current smoker 0 (0) 2 (3) Nulliparous 39 (71)* 32 (50) Early‐onset PE 23 (42) — Severe PE 38 (69) — Taking antihypertensive medication 47 (86) — GA at delivery (weeks) 37 (34–38)† 40 (39–41) Birth weight (g) 2560 (1970–3160)‡ 3645 (3363–4030) Time of postpartum evaluation (months) 13 (11.5–13) 13 (11.5–13) Data are given as median (interquartile range) or n (%). Comparison of groups: * P < 0.05;

Characteristic PE Normal Maternal age (years) 30 (26–34) 30 (28–33) GA at examination (weeks) 35 (27–37) 36 (34–37) Current smoker 0 (0) 2 (3) Nulliparous 39 (71)* 32 (50) Early‐onset PE 23 (42) — Severe PE 38 (69) — Taking antihypertensive medication 47 (86) — GA at delivery (weeks) 37 (34–38)† 40 (39–41) Birth weight (g) 2560 (1970–3160)‡ 3645 (3363–4030) Time of postpartum evaluation (months) 13 (11.5–13) 13 (11.5–13) Data are given as median (interquartile range) or n (%). Comparison of groups: * P < 0.05; † P < 0.001; ‡ P < 0.001, adjusted for gestational age (GA) at delivery. Data reused with permission from the American Heart Association22. In women with PE, BMI, SBP, DBP and MAP were all significantly higher than in women with normal pregnancy, at both inclusion and 1 year postpartum (Table  2), as described in our previous publication22. Of the women who started antihypertensive medication at PE diagnosis, most finished the treatment within a few days and all women were without antihypertensive medication within 6 weeks after delivery. None was receiving antihypertensive medication at the examination at 1 year postpartum. Table 2 Measurements of modifiable cardiovascular risk factors in women with pre‐eclampsia (PE) and with normal pregnancy at study inclusion and 1 year postpartum (PP)

In women with PE, BMI, SBP, DBP and MAP were all significantly higher than in women with normal pregnancy, at both inclusion and 1 year postpartum (Table  2), as described in our previous publication22. Of the women who started antihypertensive medication at PE diagnosis, most finished the treatment within a few days and all women were without antihypertensive medication within 6 weeks after delivery. None was receiving antihypertensive medication at the examination at 1 year postpartum. Table 2 Measurements of modifiable cardiovascular risk factors in women with pre‐eclampsia (PE) and with normal pregnancy at study inclusion and 1 year postpartum (PP) Characteristic PE Normal At inclusion (n = 55) 1 year PP (n = 48) At inclusion (n = 64) 1 year PP (n = 58) Body mass index (kg/m2) 33 (27–35)*, ‡ 27 (23–32)‡ 27 (25–30)† 23 (21–27) Systolic blood pressure (mmHg) 145 (140–151)*, ‡ 120 (115–125)* 113 (110–120)§ 110 (105–115) Diastolic blood pressure (mmHg) 91 (83–100)*, ‡ 80 (71–80)* 70 (65–75) 70 (65–75) Mean arterial pressure (mmHg) 110 (103–117)*, ‡ 93 (87–97)* 85 (80–88) 83 (78–87) Data are given as median (interquartile range). Comparisons with corresponding assessment in normal pregnancy: * P < 0.0001; ‡ P < 0.001. Comparisons with postpartum assessment in same group: † P < 0.0001; § P = 0.001. Data reused with permission from the American Heart Association22.

Characteristic PE Normal At inclusion (n = 55) 1 year PP (n = 48) At inclusion (n = 64) 1 year PP (n = 58) Body mass index (kg/m2) 33 (27–35)*, ‡ 27 (23–32)‡ 27 (25–30)† 23 (21–27) Systolic blood pressure (mmHg) 145 (140–151)*, ‡ 120 (115–125)* 113 (110–120)§ 110 (105–115) Diastolic blood pressure (mmHg) 91 (83–100)*, ‡ 80 (71–80)* 70 (65–75) 70 (65–75) Mean arterial pressure (mmHg) 110 (103–117)*, ‡ 93 (87–97)* 85 (80–88) 83 (78–87) Data are given as median (interquartile range). Comparisons with corresponding assessment in normal pregnancy: * P < 0.0001; ‡ P < 0.001. Comparisons with postpartum assessment in same group: † P < 0.0001; § P = 0.001. Data reused with permission from the American Heart Association22. At inclusion, women with PE had significantly higher levels of serum sFlt‐1, a higher sFlt‐1/PlGF ratio and significantly lower levels of serum PlGF than did women with normal pregnancy (all P < 0.0001) (Table  3). In 56% of women with PE and in 5% of normal pregnancies, serum PlGF levels were < 10 pg/mL, the detection limit of the ELISA. As described previously22, women with PE had significantly thicker CCA intima (P < 0.0001) and thinner media (P = 0.001) dimensions and a higher I/M ratio (P < 0.0001) than did women with a normal pregnancy; however, there was no difference in the conventional IMT measurement between groups (Table  3). Table 3 Serum levels of proangiogenic and antiangiogenic factors and dimensions of common carotid arterial wall layers in women with pre‐eclampsia (PE) and women with normal pregnancy at study inclusion and 1 year postpartum (PP)

At inclusion, women with PE had significantly higher levels of serum sFlt‐1, a higher sFlt‐1/PlGF ratio and significantly lower levels of serum PlGF than did women with normal pregnancy (all P < 0.0001) (Table  3). In 56% of women with PE and in 5% of normal pregnancies, serum PlGF levels were < 10 pg/mL, the detection limit of the ELISA. As described previously22, women with PE had significantly thicker CCA intima (P < 0.0001) and thinner media (P = 0.001) dimensions and a higher I/M ratio (P < 0.0001) than did women with a normal pregnancy; however, there was no difference in the conventional IMT measurement between groups (Table  3). Table 3 Serum levels of proangiogenic and antiangiogenic factors and dimensions of common carotid arterial wall layers in women with pre‐eclampsia (PE) and women with normal pregnancy at study inclusion and 1 year postpartum (PP) Variable PE Normal At inclusion (n = 55) 1 year PP (n = 48) At inclusion (n = 64) 1 year PP (n = 58) sFlt‐1 (pg/mL) 27 994 (12 876–33 463)* † ‡ 536 (367–674)¶ 3011 (1962–4509)† 231 (190–427) PlGF (pg/mL) 30 (25–48)* § 32 (19–99) 146 (70–235)† 28 (18–77) sFlt‐1/PlGF ratio 1909 (507–3139)* † ‡ 39 (27–55)* 24 (8–69) 10 (3–23) Intima thickness (mm) 0.18 (0.16–0.19)* † ‡ 0.12 (0.11–0.13)* 0.11 (0.09–0.12)† 0.08 (0.07–0.09) Media thickness (mm) 0.45 (0.37–0.54)¶ 0.48 (0.40–0.57)** 0.54 (0.46–0.62) 0.53 (0.48–0.61) Intima:media thickness ratio 0.39 (0.32–0.49)* † ‡ 0.26 (0.20–0.30)* 0.20 (0.16–0.24)† 0.15 (0.13–0.18) Intima–media thickness (mm) 0.64 (0.55–0.75) 0.60 (0.51–0.71) 0.63 (0.55–0.72) 0.62 (0.54–0.70) Data are given as median (interquartile range).

) 0.45 (0.37–0.54)¶ 0.48 (0.40–0.57)** 0.54 (0.46–0.62) 0.53 (0.48–0.61) Intima:media thickness ratio 0.39 (0.32–0.49)* † ‡ 0.26 (0.20–0.30)* 0.20 (0.16–0.24)† 0.15 (0.13–0.18) Intima–media thickness (mm) 0.64 (0.55–0.75) 0.60 (0.51–0.71) 0.63 (0.55–0.72) 0.62 (0.54–0.70) Data are given as median (interquartile range). Comparisons with corresponding assessment in normal pregnancy: * P < 0.0001; ¶ P = 0.001; ** P < 0.05. Comparisons with postpartum assessment in the same group: † P < 0.0001. Comparisons with corresponding assessment in normal pregnancy, after adjustment for body mass index, blood pressure, smoking status and family history of cardiovascular disease: ‡ P < 0.0001; § P = 0.001. Data on arterial wall layers reused with permission from the American Heart Association22. PlGF, placental growth factor; sFlt‐1, soluble fms‐like tyrosine kinase‐1. We found a clear reduction in serum levels of sFlt‐1 and sFlt‐1/PlGF ratio values from pregnancy to the postpartum assessment, in both PE and normal pregnancies. There were still significant group differences in sFlt‐1 levels and the sFlt‐1/PlGF ratio (P = 0.001 and P < 0.0001, respectively) at analyses at 1 year postpartum (Table  3).

reduction in serum levels of sFlt‐1 and sFlt‐1/PlGF ratio values from pregnancy to the postpartum assessment, in both PE and normal pregnancies. There were still significant group differences in sFlt‐1 levels and the sFlt‐1/PlGF ratio (P = 0.001 and P < 0.0001, respectively) at analyses at 1 year postpartum (Table  3). At the time of inclusion, there were strong positive correlations between both serum sFlt‐1 and the sFlt‐1/PlGF ratio and intima thickness (rs = 0.51 and 0.63, respectively; both P < 0.0001) (Figure  2a) and the I/M ratio (rs = 0.50 and 0.61, respectively; both P < 0.0001) (Figure  2b) for the combined group of PE and normal pregnancies. Similarly, we found inverse correlations between PlGF and intima thickness and I/M ratio (rs = −0.44 and −0.47, respectively; both P < 0.0001). After adjusting for common confounding factors (BMI, blood pressure, smoking status and family history of CVD), angiogenic factors and arterial wall layer dimensions still differed significantly between PE and normal pregnancy (Table  3). At 1 year postpartum, there were still significant positive correlations between sFlt‐1 and intima thickness (rs = 0.38, P = 0.007) and between the sFlt‐1/PlGF ratio and CCA intima thickness and I/M ratio (rs = 0.48 and 0.41; P < 0.0001 and 0.003, respectively). Similarly, negative correlations were found between PlGF and CCA intima thickness and I/M ratio (rs = −0.21 and −0.21; P = 0.04 and 0.03, respectively) (data not shown). When we analyzed findings of the PE and normal pregnancy groups separately, we found no correlation between levels of angiogenic factors and arterial wall layer dimensions. There were no correlations between levels of sFlt‐1 and PlGF, and the sFlt‐1/PlGF ratio vs CCA‐IMT at inclusion (Figure  2c) or at 1 year postpartum.

en we analyzed findings of the PE and normal pregnancy groups separately, we found no correlation between levels of angiogenic factors and arterial wall layer dimensions. There were no correlations between levels of sFlt‐1 and PlGF, and the sFlt‐1/PlGF ratio vs CCA‐IMT at inclusion (Figure  2c) or at 1 year postpartum. For the combined groups at inclusion, we found that women with higher BMI, SBP, DBP and MAP often had higher sFlt‐1 levels and sFlt‐1/PlGF ratios and lower levels of PlGF. Similarly, we also found that these women with higher BMI, SBP, DBP and MAP often had thicker intima and thinner media dimensions and a higher I/M ratio of the CCA22. No correlations were found between maternal age vs angiogenic factors and arterial wall layer dimensions (Table  4). At 1 year postpartum, BMI and blood pressure had decreased in both groups compared with during pregnancy and no significant correlations were found between BMI and blood pressure vs angiogenic factors. However, there were still positive correlations between BMI and blood pressure vs arterial wall layer dimensions in postpartum analyses (data not shown for postpartum analyses). Table 4 Associations between cardiovascular risk factors vs angiogenic factors and dimensions of common carotid artery (CCA) wall layers at study inclusion in women with pre‐eclampsia or normal pregnancy Cardiovascular risk factors Angiogenic factors rs CCA wall layer dimensions rs Body mass index (kg/m2) sFlt‐1 0.30* Intima 0.34† PlGF −0.40* Media −0.24‡ sFlt‐1/PlGF 0.38* Intima:media ratio 0.37* Systolic blood pressure (mmHg) sFlt‐1 0.61† Intima 0.70*

Table 4 Associations between cardiovascular risk factors vs angiogenic factors and dimensions of common carotid artery (CCA) wall layers at study inclusion in women with pre‐eclampsia or normal pregnancy Cardiovascular risk factors Angiogenic factors rs CCA wall layer dimensions rs Body mass index (kg/m2) sFlt‐1 0.30* Intima 0.34† PlGF −0.40* Media −0.24‡ sFlt‐1/PlGF 0.38* Intima:media ratio 0.37* Systolic blood pressure (mmHg) sFlt‐1 0.61† Intima 0.70* PlGF −0.66* Media −0.17 sFlt‐1/PlGF 0.71* Intima:media ratio 0.60* Diastolic blood pressure (mmHg) sFlt‐1 0.60* Intima 0.65* PlGF −0.66* Media −0.18 sFlt‐1/PlGF 0.70* Intima:media ratio 0.55* Mean arterial pressure (mmHg) sFlt‐1 0.62* Intima 0.68* PlGF −0.67* Media −0.17 sFlt‐1/PlGF 0.72* Intima:media ratio 0.58* Age (years) sFlt‐1 −0.05 Intima 0.04 PlGF −0.025 Media 0.03 sFlt‐1/PlGF −0.009 Intima:media ratio 0.01 * P < 0.0001; † P = 0.001; ‡ P = 0.008. PlGF, placental growth factor; sFlt‐1, soluble fms‐like tyrosine kinase‐1. At inclusion, among women with PE, we found no differences in serum levels of sFlt‐1 and PlGF and the sFlt‐1/PlGF ratio between early‐ and late‐onset PE, women who were on antihypertensive medication vs those who were not, women with CVD heredity vs those without (except for PlGF, P = 0.04), women who delivered preterm vs at term or women who delivered an SGA vs appropriate‐for‐gestational age infant (data not shown).

sFlt‐1/PlGF ratio between early‐ and late‐onset PE, women who were on antihypertensive medication vs those who were not, women with CVD heredity vs those without (except for PlGF, P = 0.04), women who delivered preterm vs at term or women who delivered an SGA vs appropriate‐for‐gestational age infant (data not shown). DISCUSSION In women with PE, we found substantially higher levels of serum sFlt‐1, a higher sFlt‐1/PlGF ratio and lower levels of serum PlGF compared with those in women with normal pregnancy, at both inclusion and 1 year postpartum. We also found that serum levels of sFlt‐1 and the sFlt‐1/PlGF ratio were positively associated with CCA intima thickness and I/M ratio, and negatively associated with CCA media thickness, i.e. signs of arterial aging. Women with elevated BMI and blood pressure often had higher levels of antiangiogenic factors and negatively affected arterial wall layer dimensions.

the sFlt‐1/PlGF ratio were positively associated with CCA intima thickness and I/M ratio, and negatively associated with CCA media thickness, i.e. signs of arterial aging. Women with elevated BMI and blood pressure often had higher levels of antiangiogenic factors and negatively affected arterial wall layer dimensions. sFlt‐1 is a splice variant of vascular endothelial growth factor 1 (Flt‐1) and is produced mainly by the placenta7 and endothelial cells28, 29. sFlt‐1 is a strong inhibitor of angiogenic activity by binding to and inactivating the proangiogenic factor PlGF30. High serum levels of sFlt‐1 and low levels of PlGF predict and correlate with the onset of clinical signs of PE9, 31. We and others9, 10, 31 have shown that PE is associated with higher circulating levels of sFlt‐1 and lower levels of PlGF compared with women with normal pregnancy. In the present study we found that, after 1 year, women in the PE group still had higher levels of sFlt‐1 compared with normal pregnancies. Two earlier studies have revealed persisting elevated sFlt‐1 levels after delivery in women who had PE, but the investigators examined sFlt‐1 levels at 2 days32 and 1 week postpartum33. Our finding of elevated levels of sFlt‐1 at 1 year postpartum could be explained by the presence of extraplacental production of sFlt‐128, 29 and persistent endothelial dysfunction in women with previous/recent PE34, 35, 36.

n who had PE, but the investigators examined sFlt‐1 levels at 2 days32 and 1 week postpartum33. Our finding of elevated levels of sFlt‐1 at 1 year postpartum could be explained by the presence of extraplacental production of sFlt‐128, 29 and persistent endothelial dysfunction in women with previous/recent PE34, 35, 36. Probably due to the small sample size and the risk of Type II error, we could not find any significant differences in sFlt‐1 and PlGF levels between early‐ vs late‐onset PE, preterm vs term delivery or SGA vs appropriate‐for‐gestational‐age infant births. Previous studies have found pronounced alterations in sFlt‐1 and PlGF levels in early‐onset compared with late‐onset PE37, 38, and in pregnancies with preterm delivery39 and SGA infants40.

and PlGF levels between early‐ vs late‐onset PE, preterm vs term delivery or SGA vs appropriate‐for‐gestational‐age infant births. Previous studies have found pronounced alterations in sFlt‐1 and PlGF levels in early‐onset compared with late‐onset PE37, 38, and in pregnancies with preterm delivery39 and SGA infants40. PE is thought to be a ‘stress test’ with regard to future risk of CVD41. Recently, two large meta‐analyses, by McDonald et al.18 and Bellamy et al.19, showed that women with PE have higher risks of coronary heart disease and stroke later in life. The serum concentration of sFlt‐1 has been shown to be increased in individuals with acute myocardial infarction compared with those without, and it is a good predictor of development of heart failure in patients with coronary heart disease14, 15, 16. Further, in an earlier study, we showed that women with PE had thicker intima and thinner media dimensions and a higher I/M ratio compared with women with normal pregnancy, both during pregnancy and at 1 year postpartum22. Our current findings of highly significant and logical correlations of sFlt‐1 and PlGF and the sFlt‐1/PlGF ratio with signs of arterial aging are in line with the findings of McDonald et al.18 and Bellamy et al.19, and support our previous findings22.

normal pregnancy, both during pregnancy and at 1 year postpartum22. Our current findings of highly significant and logical correlations of sFlt‐1 and PlGF and the sFlt‐1/PlGF ratio with signs of arterial aging are in line with the findings of McDonald et al.18 and Bellamy et al.19, and support our previous findings22. High blood pressure and obesity are two of the modifiable risk factors in regard to the development of CVD8. We found that, at inclusion, BMI and blood pressure were higher in women with PE than in those with normal pregnancy22. At postpartum analysis, BMI and blood pressure had decreased in both PE and normal pregnancy groups, but there was still a difference between the groups. These postpartum differences in BMI and blood pressure, together with persistent positive correlations between BMI and blood pressure vs arterial wall layer thicknesses indicate that the effects of PE on the cardiovascular system are longstanding. These findings are in line with our main findings of highly significant positive correlations between sFlt‐1 levels and sFlt‐1/PlGF ratio vs CCA intima thickness and I/M ratio, at both inclusion and 1 year postpartum.

wall layer thicknesses indicate that the effects of PE on the cardiovascular system are longstanding. These findings are in line with our main findings of highly significant positive correlations between sFlt‐1 levels and sFlt‐1/PlGF ratio vs CCA intima thickness and I/M ratio, at both inclusion and 1 year postpartum. Because of substantial overlap in levels of serum sFlt‐1 (Figure  2) and PlGF (data not shown in figure) between women with PE and those with normal pregnancy, we tested the correlations between serum levels of sFlt‐1 and PlGF and sFlt‐1/PlGF ratio vs arterial wall layer dimensions in the groups combined. Previous studies have shown that normal pregnancy represents a state of mild systemic inflammation3, 4 and in an earlier study we showed that normal pregnancy is also associated with increased levels of sFlt‐1 compared with those in non‐pregnant women37. Further, our group has shown previously that women with normal pregnancy, who are older, with higher BMI and blood pressure, also have negatively affected arterial wall layer dimensions during pregnancy22.

t normal pregnancy is also associated with increased levels of sFlt‐1 compared with those in non‐pregnant women37. Further, our group has shown previously that women with normal pregnancy, who are older, with higher BMI and blood pressure, also have negatively affected arterial wall layer dimensions during pregnancy22. A strength of our study is that we obtained serum levels of sFlt‐1 and PlGF and values of CCA wall layer measurements both during pregnancy and at about 1 year after delivery, which permitted analysis of postpartum changes. We have found repeatedly that the use of CCA intima thickness and I/M ratio is superior to that of IMT for imaging the effects of vascular aging and CVD27 and long‐term estrogen therapy26. In addition, our unpublished data indicate that the method correctly images the expected vascular benefits of menopausal hormone therapy26, which was not possible when tested in very large randomized controlled trials using CCA‐IMT42, 43. A limitation of our study is the relatively small sample size with the associated potential risk of Type II error.

indicate that the method correctly images the expected vascular benefits of menopausal hormone therapy26, which was not possible when tested in very large randomized controlled trials using CCA‐IMT42, 43. A limitation of our study is the relatively small sample size with the associated potential risk of Type II error. In conclusion, we have shown that levels of serum sFlt‐1 and sFlt‐1/PlGF ratio are associated with signs of arterial aging, as estimated by high‐frequency ultrasonography, both during pregnancy and at 1 year postpartum. Further, we have shown that levels of sFlt‐1 and the sFlt‐1/PlGF ratio are associated with two of the modifiable cardiovascular risk factors during pregnancy. These parameters therefore seem to reflect the degree of vascular damage during pregnancy and also at 1 year postpartum. In addition, our data confirm previous findings of higher serum sFlt‐1 and lower PlGF levels in women with PE compared with women with normal pregnancy and add new information concerning a persistent difference between the groups at 1 year postpartum. Further study is needed to investigate the long‐term effects of PE‐related antiangiogenic factors on arterial wall layers. ACKNOWLEDGMENTS We are grateful to Lars Berglund PhD (UCR, Uppsala, Sweden) for expert statistical advice. This study was supported by grants from the Selanders Foundation, Thuréus Foundation, Research Council of Uppsala County Council and ALF funding from Uppsala University Hospital and the Swedish Research Council (A.‐K.W., project number: 2014–3561).

INTRODUCTION Since its introduction in 1981, intrauterine intravascular blood transfusion (IUT) has become the cornerstone of treatment for fetal anemia in pregnancies complicated by red‐cell alloimmunization1. In experienced hands, it is nowadays considered a safe procedure, significantly improving perinatal outcome in fetuses with severe anemia2, 3. One way to improve fetal outcome further is to minimize the occurrence of procedure‐related (PR) complications associated with IUT. A PR fetal‐loss rate of approximately 2% per procedure was reported in a review by Schumacher and Moise4. However, the small studies included in their review used a variety of definitions for complications and treatment techniques. In 2005, we reported PR complications and fetal‐loss rates of 3.1% and 1.6% per procedure, respectively, in a single‐center series of 740 IUTs5. In our study, transamniotic ‘free loop’ needling, inadvertent arterial puncture and refraining from the use of fetal paralysis were identified as risk factors for adverse outcome5. Apart from the technical aspects, operator and team experience are known to be of utmost importance for performing successful and safe IUTs 6. In recent years, a few smaller studies have been performed on this subject, revealing no relevant new insights or tools to improve perinatal outcome further after IUT7, 8.

ome5. Apart from the technical aspects, operator and team experience are known to be of utmost importance for performing successful and safe IUTs 6. In recent years, a few smaller studies have been performed on this subject, revealing no relevant new insights or tools to improve perinatal outcome further after IUT7, 8. The present study aimed to evaluate PR complications and perinatal loss rates after IUT, including assessment of their change over time, over a period of nearly three decades in a national single‐center cohort, in order to identify factors leading to improved outcome. METHODS Patients We included all patients treated with IUT between January 1988 and January 2015 at the national referral center for fetal therapy in Leiden, The Netherlands, for fetal anemia caused by red‐blood‐cell alloimmunization. The findings of IUTs performed between 1988 and 2001 have been analyzed and published previously5.

nts We included all patients treated with IUT between January 1988 and January 2015 at the national referral center for fetal therapy in Leiden, The Netherlands, for fetal anemia caused by red‐blood‐cell alloimmunization. The findings of IUTs performed between 1988 and 2001 have been analyzed and published previously5. Patient data, technical aspects of IUT and complications were collected from our custom‐built electronic Rhesus database. As early IUT is known to be associated with higher perinatal loss rates9, 10, 11, 12, we compared outcomes after IUTs performed before and after 20 weeks' gestation to determine whether expected improvements in survival also apply to very young anemic fetuses. To identify changes in procedural techniques and perinatal outcome, pregnancies were divided into two cohorts according to year of procedure. The first cohort included patients with a first IUT before January 2001, described previously5, and the second cohort included patients with a first IUT performed from 1 January 2001 onwards. This policy was chosen as it was assumed that the findings of our first study in 20055 may have resulted in gradual changes in transfusion techniques and therefore also in perinatal survival.

ary 2001, described previously5, and the second cohort included patients with a first IUT performed from 1 January 2001 onwards. This policy was chosen as it was assumed that the findings of our first study in 20055 may have resulted in gradual changes in transfusion techniques and therefore also in perinatal survival. Complications after IUT were classified independently into PR or non‐procedure related (NPR)5 by two operators (I.L.v.K. and D.O.). In summary, fetal condition before IUT was assessed by ultrasound (presence of hydrops, biophysical profile), fetal heart rate tracing in fetuses > 26 weeks' gestation and blood gas analysis of the fetal blood sample obtained before transfusion (a blood pH of ≤ 7.25 was considered a sign of fetal compromise)5. If a complication occurred during or after a complicated procedure in a fetus with a reasonable condition prior to IUT based on the abovementioned findings, the complication was considered to be related to the procedure i.e. PR. In fetuses with an unfavorable condition prior to an uncomplicated procedure, the complication was classified as not related to the procedure i.e. NPR. If the experts could not agree on classification, the complication remained unclassified. The following complications were taken into account: rupture of membranes or preterm delivery within 7 days after IUT, if occurring before 34 weeks of gestation; and intrauterine infection and fetal distress resulting in either emergency Cesarean section (CS) within 24 h after IUT or fetal or neonatal death.

Complications after IUT were classified independently into PR or non‐procedure related (NPR)5 by two operators (I.L.v.K. and D.O.). In summary, fetal condition before IUT was assessed by ultrasound (presence of hydrops, biophysical profile), fetal heart rate tracing in fetuses > 26 weeks' gestation and blood gas analysis of the fetal blood sample obtained before transfusion (a blood pH of ≤ 7.25 was considered a sign of fetal compromise)5. If a complication occurred during or after a complicated procedure in a fetus with a reasonable condition prior to IUT based on the abovementioned findings, the complication was considered to be related to the procedure i.e. PR. In fetuses with an unfavorable condition prior to an uncomplicated procedure, the complication was classified as not related to the procedure i.e. NPR. If the experts could not agree on classification, the complication remained unclassified. The following complications were taken into account: rupture of membranes or preterm delivery within 7 days after IUT, if occurring before 34 weeks of gestation; and intrauterine infection and fetal distress resulting in either emergency Cesarean section (CS) within 24 h after IUT or fetal or neonatal death. Primary outcomes assessed were perinatal survival and PR complications. Furthermore, we assessed primary antibody type, gestational age at first IUT, fetal hemoglobin concentration and presence of hydrops at first IUT, procedure access site and other technical details of IUT, number of transfusions per fetus and gestational age at delivery.

sed were perinatal survival and PR complications. Furthermore, we assessed primary antibody type, gestational age at first IUT, fetal hemoglobin concentration and presence of hydrops at first IUT, procedure access site and other technical details of IUT, number of transfusions per fetus and gestational age at delivery. Diagnostics Patients with red‐cell alloimmunization were referred, according to national guidelines, to our center in Leiden, The Netherlands, which has served as the national referral center for fetal therapy since 1965. Indication for referral is based on type and titer of antibody, level of antibody‐dependent cell‐mediated cytotoxicity assay13 and obstetric history. In the first decade of the study period, amniotic fluid delta optical density measurements at 450 nm were used to assess the likelihood of fetal anemia. In later years, the peak systolic velocity in the fetal middle cerebral artery (MCA‐PSV) was used to determine the optimal timing of first and subsequent IUTs14, 15, 16. Cordocentesis was performed if MCA‐PSV exceeded 1.5 multiples of the median and/or if signs of hydrops were detected on ultrasound, and was followed by transfusion if the fetal blood sample showed fetal anemia. Intrauterine intravascular blood transfusion technique Our IUT technique has been described comprehensively in a previous publication3. No routine antibiotic prophylaxis or corticosteroid was administered prior to IUT. In nearly all recent cases, atracurium was given intravenously (or intramuscularly) to the fetus prior to transfusion to cause fetal paralysis.

on technique Our IUT technique has been described comprehensively in a previous publication3. No routine antibiotic prophylaxis or corticosteroid was administered prior to IUT. In nearly all recent cases, atracurium was given intravenously (or intramuscularly) to the fetus prior to transfusion to cause fetal paralysis. IUT was carried out under aseptic conditions, using a 20‐ or 22‐G needle. In this study, all IUT attempts were made intravascularly. As a result of our prior study5, fetal paralysis was applied more often and known risk factors for PR complications, such as transamniotic needling (also known as ‘free loop’ needling) and arterial puncture, were avoided more consciously in the second half of the study. A tailored mode of transfusion was chosen depending on the fetal anatomy, preferably transfusing into the placental cord insertion in the case of an anterior placenta and into the fetal intrahepatic umbilical vein in the case of a posterior placenta, often in combination with additional intraperitoneal transfusion. For both time cohorts, a maximum of four experienced operators were involved, all capable of applying different transfusion techniques. IUT was considered to be successful if more blood than was acquired for fetal blood sampling was transfused to the fetus, as verified by ultrasound. Fetal condition was monitored before, during and after transfusion. If the condition of the mother and fetus was satisfactory, patients were discharged within 6 h after IUT.

IUT was carried out under aseptic conditions, using a 20‐ or 22‐G needle. In this study, all IUT attempts were made intravascularly. As a result of our prior study5, fetal paralysis was applied more often and known risk factors for PR complications, such as transamniotic needling (also known as ‘free loop’ needling) and arterial puncture, were avoided more consciously in the second half of the study. A tailored mode of transfusion was chosen depending on the fetal anatomy, preferably transfusing into the placental cord insertion in the case of an anterior placenta and into the fetal intrahepatic umbilical vein in the case of a posterior placenta, often in combination with additional intraperitoneal transfusion. For both time cohorts, a maximum of four experienced operators were involved, all capable of applying different transfusion techniques. IUT was considered to be successful if more blood than was acquired for fetal blood sampling was transfused to the fetus, as verified by ultrasound. Fetal condition was monitored before, during and after transfusion. If the condition of the mother and fetus was satisfactory, patients were discharged within 6 h after IUT. Statistical analysis To compare proportions, Fisher's exact test (or Pearson's chi‐square test, when appropriate), binary logistic regression (Wald test) or Mann–Whitney U‐test was used. The independent t‐test was used for comparison of means. All variables with P‐value of ≤ 0.15 in univariate analysis were included in a multiple logistic regression model to identify possible independent risk factors for severe PR complications (emergency CS or death). A P‐value of < 0.05 was considered statistically significant.

dent t‐test was used for comparison of means. All variables with P‐value of ≤ 0.15 in univariate analysis were included in a multiple logistic regression model to identify possible independent risk factors for severe PR complications (emergency CS or death). A P‐value of < 0.05 was considered statistically significant. RESULTS During the 27‐year study period, 595 fetuses in 587 pregnancies of 497 women received a total of 1685 IUTs, of which 740 were described previously5. In eight twin pregnancies, both twins had anemia as a result of red‐cell immunization and received IUTs. In one additional twin pregnancy, one fetus was affected and was included in this study; the cotwin was Rhesus D (RhD) negative. Six singleton pregnancies, in which fetal death occurred from causes unrelated to red‐cell alloimmunization, were excluded, as described in detail previously5. Therefore, 589 fetuses were treated with a total of 1678 IUTs in 581 pregnancies of 491 women. Of these, 741 procedures were performed in 255 fetuses before 2001 and 937 procedures were performed in 334 fetuses from 2001 onwards.

unrelated to red‐cell alloimmunization, were excluded, as described in detail previously5. Therefore, 589 fetuses were treated with a total of 1678 IUTs in 581 pregnancies of 491 women. Of these, 741 procedures were performed in 255 fetuses before 2001 and 937 procedures were performed in 334 fetuses from 2001 onwards. Overall perinatal survival was 93.4%, significantly increasing from 88.6% in the first time‐cohort to 97.0% in the second time‐cohort (odds ratio (OR), 4.2 (95% CI, 2.0–8.7), P < 0.001). Characteristics of the study population in both time‐cohorts are summarized in Table 1. The main type of immunization was for RhD, although this significantly decreased from 85.1% before 2001 to 76.3% from 2001 onwards (P = 0.009), whereas the proportion of Kell immunization increased (9.8% to 15.9%, P = 0.037, Table 1). In the second time‐cohort, four patients were treated with intravenous immunoglobulin prior to the first transfusion. Table 1 Characteristics of 1678 intrauterine intravascular blood transfusions (IUTs) in 589 fetuses with anemia caused by red‐cell alloimmunization, according to study period in which procedure was performed Characteristic 1988–2000 (n = 255 fetuses/741 IUTs) 2001–2015 (n = 334 fetuses/937 IUTs) P Primary immunization against: Rhesus D 217 (85.1) 255 (76.3) 0.009 Kell 25 (9.8) 53 (15.9) 0.037 Other* 13 (5.1) 26 (7.8) 0.242 GA at first IUT (weeks) 27 (17 to 36) 27 (16 to 35) 0.787 Hydrops at first IUT 97 (38.0) 43 (12.9) < 0.001 Hemoglobin at first IUT (g/dL) 4.8 (1.1 to 13.2) 6.3 (1.5 to 12.9) < 0.001

Characteristic 1988–2000 (n = 255 fetuses/741 IUTs) 2001–2015 (n = 334 fetuses/937 IUTs) P Primary immunization against: Rhesus D 217 (85.1) 255 (76.3) 0.009 Kell 25 (9.8) 53 (15.9) 0.037 Other* 13 (5.1) 26 (7.8) 0.242 GA at first IUT (weeks) 27 (17 to 36) 27 (16 to 35) 0.787 Hydrops at first IUT 97 (38.0) 43 (12.9) < 0.001 Hemoglobin at first IUT (g/dL) 4.8 (1.1 to 13.2) 6.3 (1.5 to 12.9) < 0.001 Z‐hemoglobin at first IUT† −8.3 (−12.2 to −0.24) −6.9 (−11.7 to −0.5) <0.001 Δ Hemoglobin (after IUT − before IUT) (g/dL) 4.5 (−0.5 to 11.2) 4.4 (0.5 to 9.2) 0.039 Number of IUTs per fetus 3 (1 to 7) 3 (1 to 6) 0.337 GA at delivery of liveborn (weeks) 37 (30 to 39) 36 (28 to 39) < 0.001 Data are given as n (%) or median (range). * Rhesus c, E or e, Duffy (Fya), Kidd (Jka), rare or low‐frequency antigens. † Number of SDs from median concentration for gestational age. GA, gestational age. Complications In a total of 1678 IUTs, 69 complications occurred. Forty‐four of these were considered as being directly related to the procedure (PR). In eight patients, one PR complication led to another (five emergency CSs followed by perinatal death, two intrauterine infections followed by perinatal death and one intrauterine infection followed by emergency CS). After correction for this, the actual PR complication rates for the total cohort over 27 years were 6.1% per fetus and 2.1% per procedure. Survival and PR complications are listed in Table 2.

rinatal death, two intrauterine infections followed by perinatal death and one intrauterine infection followed by emergency CS). After correction for this, the actual PR complication rates for the total cohort over 27 years were 6.1% per fetus and 2.1% per procedure. Survival and PR complications are listed in Table 2. Table 2 Outcome and procedure‐related complications after 1678 intrauterine intravascular blood transfusions (IUTs) in 589 fetuses with anemia caused by red‐cell alloimmunization, according to study period in which procedure was performed Outcome 1988–2000 (n = 255 fetuses/ 741 IUTs) 2001–2015 (n = 334 fetuses/ 937 IUTs) OR (95% CI) P Survival (n (%))* 226 (88.6) 324 (97.0) 4.16 (2.0–8.7) < 0.001 Procedure‐related complication (n) 32 12 Per fetus (n (%))† 25 (9.8) 11 (3.3) 0.31 (0.2–0.7) 0.001 Per procedure (n (%))† 25 (3.4) 11 (1.2) 0.34 (0.2–0.7) 0.003 Procedure‐related PPROM (n) 1 1 Per fetus (%) 0.4 0.3 0.76 (0.0–12.3) 1.000 Per procedure (%) 0.1 0.1 0.79 (0.0–12.7) 1.000 Procedure‐related infection (n) 2 1 Per fetus (%) 0.8 0.3 0.38 (0.0–4.2) 0.581 Per procedure (%) 0.3 0.1 0.40 (0.0–4.4) 0.587 Procedure‐related emergency CS (n) 17 4 Per fetus (%) 6.7 1.2 0.17 (0.1–0.5) < 0.001 Per procedure (%) 2.3 0.4 0.18 (0.1–0.5) < 0.001 Procedure‐related loss (n) 12 6 Per fetus (%) 4.7 1.8 0.37 (0.1–1.0) 0.053 Per procedure (%) 1.6 0.6 0.39 (0.1–1.0) 0.059 * Alive at discharge from tertiary center. † Actual number and rate (eight patients had two interrelated complications). CS, Cesarean section; OR, odds ratio; PPROM, preterm prelabor rupture of membranes.

Survival (n (%))* 226 (88.6) 324 (97.0) 4.16 (2.0–8.7) < 0.001 Procedure‐related complication (n) 32 12 Per fetus (n (%))† 25 (9.8) 11 (3.3) 0.31 (0.2–0.7) 0.001 Per procedure (n (%))† 25 (3.4) 11 (1.2) 0.34 (0.2–0.7) 0.003 Procedure‐related PPROM (n) 1 1 Per fetus (%) 0.4 0.3 0.76 (0.0–12.3) 1.000 Per procedure (%) 0.1 0.1 0.79 (0.0–12.7) 1.000 Procedure‐related infection (n) 2 1 Per fetus (%) 0.8 0.3 0.38 (0.0–4.2) 0.581 Per procedure (%) 0.3 0.1 0.40 (0.0–4.4) 0.587 Procedure‐related emergency CS (n) 17 4 Per fetus (%) 6.7 1.2 0.17 (0.1–0.5) < 0.001 Per procedure (%) 2.3 0.4 0.18 (0.1–0.5) < 0.001 Procedure‐related loss (n) 12 6 Per fetus (%) 4.7 1.8 0.37 (0.1–1.0) 0.053 Per procedure (%) 1.6 0.6 0.39 (0.1–1.0) 0.059 * Alive at discharge from tertiary center. † Actual number and rate (eight patients had two interrelated complications). CS, Cesarean section; OR, odds ratio; PPROM, preterm prelabor rupture of membranes. In the first time‐cohort, there were 51 complications (seven women each had two complications; actual complication rate = 5.9% per procedure) and in the second time‐cohort there were 18 (one women had two complications; actual complication rate = 1.7% per procedure) (P < 0.001). Compared with the first cohort, the incidence of PR complications significantly declined in the second cohort, from 9.8% to 3.3% per fetus (OR, 0.3 (95% CI, 0.2–0.7), P = 0.001) and from 3.4% to 1.2% per procedure (OR, 0.3 (95% CI, 0.2–0.7), P = 0.003). The risk of PR perinatal loss decreased over time from 4.7% to 1.8% per fetus and from 1.6% to 0.6% per procedure (Table 2).

cations significantly declined in the second cohort, from 9.8% to 3.3% per fetus (OR, 0.3 (95% CI, 0.2–0.7), P = 0.001) and from 3.4% to 1.2% per procedure (OR, 0.3 (95% CI, 0.2–0.7), P = 0.003). The risk of PR perinatal loss decreased over time from 4.7% to 1.8% per fetus and from 1.6% to 0.6% per procedure (Table 2). Preterm prelabor rupture of membranes and preterm delivery In three cases, preterm prelabor rupture of membranes (PPROM) occurred within 7 days after transfusion, leading to preterm delivery before 34 weeks. One PR and one NPR classified PPROM are described in more detail in our previously published cohort study5. In the second time‐cohort, PPROM occurred in one case, the day after the first IUT at 30 weeks' gestation. The baby was liveborn 8 days later. As it took three attempts to complete the transfusion successfully, this complication was classified as PR. The PR‐PPROM rate was thus low in both cohorts and did not differ significantly (Table 2). Infection Three cases of culture‐proven intrauterine infection with Escherichia coli were observed, and all three were classified as PR. Two cases were part of our previously published cohort with IUTs before 20015. A third case occurred in the second cohort after IUT at 18 weeks. The infection led to fetal loss and was considered to be PR. The decrease in PR infections over time, from 0.8% to 0.3% per fetus and from 0.3% to 0.1% per procedure, was not significant (Table 2).

our previously published cohort with IUTs before 20015. A third case occurred in the second cohort after IUT at 18 weeks. The infection led to fetal loss and was considered to be PR. The decrease in PR infections over time, from 0.8% to 0.3% per fetus and from 0.3% to 0.1% per procedure, was not significant (Table 2). Emergency Cesarean section Within 24 h after IUT, 24 fetuses were delivered by emergency CS for fetal distress, which was considered PR in 21 cases. Five children died subsequently; all deaths were PR and have been described previously5. As fetal condition in two of the six cases in the second cohort was unfavorable prior to an uncomplicated IUT, these complications were considered NPR. One case of PR intrauterine infection resulted in emergency CS5. In one case, emergency CS was performed at 34 weeks' gestation for persistent tachycardia after transfusion, probably triggered by volume overload, and was considered as PR. The three remaining cases of CS at 32, 34 and 35 weeks were all classified as PR. All neonates in the second cohort survived after emergency CS. In summary, the occurrence of PR emergency CS decreased from 6.7% in the first cohort to 1.2% in the second cohort (P < 0.001) and from 2.3% to 0.4% per procedure (P < 0.001).

The three remaining cases of CS at 32, 34 and 35 weeks were all classified as PR. All neonates in the second cohort survived after emergency CS. In summary, the occurrence of PR emergency CS decreased from 6.7% in the first cohort to 1.2% in the second cohort (P < 0.001) and from 2.3% to 0.4% per procedure (P < 0.001). Fetal or neonatal death During the study period, a total of 39 cases of fetal or neonatal death occurred after IUT, of which 10 occurred from 2001 onwards. In this second cohort, one patient with RhD immunization was treated initially with (a possibly incomplete) interstitial laser for twin reversed arterial perfusion syndrome and fetal death was detected the day after the third uncomplicated IUT. Because of the complexity of this case, this complication could not be clearly classified as PR or NPR. Three of the nine remaining cases of loss were considered to be NPR. One NPR loss occurred after the decision to stop intrauterine treatment, as a result of refractory severe hydrops that was diagnosed relatively late and was caused by antibodies against a low‐frequency antigen. In the two other cases, fetal death was detected 19 and 22 days after an uncomplicated IUT and, because of the time lapse between the procedure and the death, these deaths were considered as being NPR.

actory severe hydrops that was diagnosed relatively late and was caused by antibodies against a low‐frequency antigen. In the two other cases, fetal death was detected 19 and 22 days after an uncomplicated IUT and, because of the time lapse between the procedure and the death, these deaths were considered as being NPR. Therefore, a total of six PR losses occurred in the second cohort, including one with E. coli infection that was described earlier. In three patients, multiple attempts at intravenous access led to bradycardia during the procedure, leading to fetal loss directly, and at 1 and 8 days later. Two other PR fetal losses occurred within 1 week after IUTs at 16 and 26 weeks in fetuses with a reasonable condition prior to IUT. We thus saw a decrease in PR death rates from 4.7% to 1.8% per fetus (P = 0.053) and from 1.6% to 0.6% per procedure (P = 0.059). Fetal death occurred after eight (17.0%) of 47 IUTs performed before 20 weeks. Four of these were classified as NPR and four as PR, one of the latter was preceded by infection. Fetal demise in the total cohort occurred more often before 20 weeks than after 20 weeks, accounting for both NPR (8.5% vs 1.0% per procedure; P = 0.002) and PR (8.5% vs 0.9%; P = 0.001) fetal death. No significant difference was found in PR fetal‐loss rate before 20 weeks between the first and the second time‐cohort (P = 0.083).

e in the total cohort occurred more often before 20 weeks than after 20 weeks, accounting for both NPR (8.5% vs 1.0% per procedure; P = 0.002) and PR (8.5% vs 0.9%; P = 0.001) fetal death. No significant difference was found in PR fetal‐loss rate before 20 weeks between the first and the second time‐cohort (P = 0.083). Technical details In the second time‐cohort, significantly more procedures were completed successfully than in the first time‐cohort (96.8% vs 99.0%, P = 0.001). The number of attempts per procedure decreased (median, 1 (range, 1–7) vs 1 (range, 1–5), P < 0.001). From 2001 onward, the fetal liver was the most frequently chosen procedure access site (48.0% in the second cohort vs 13.8% in the first cohort, P < 0.001). Trends in procedure access sites over time are presented in Figure 1. Figure 1 Trends in procedure access sites for intrauterine intravascular blood transfusion between January 1988 and January 2015. , liver (plus intraperitoneal); , placental cord insertion; , transamniotic venous; , arterial (cord insertion or transamniotic); , intraperitoneal; , unknown vessel, heart, chorionic vein. UOG-17319-FIG-0001-bFetal paralysis was applied significantly more frequently (97.8% vs 81.3%, P < 0.001) and transamniotic needling in a free loop of cord was performed less frequently (3.7% vs 33.1%, P < 0.001) from 2001 onwards compared with before 2001. No arterial punctures were performed after 2001 (0.0% vs 3.2%, P < 0.001).

G-0001-bFetal paralysis was applied significantly more frequently (97.8% vs 81.3%, P < 0.001) and transamniotic needling in a free loop of cord was performed less frequently (3.7% vs 33.1%, P < 0.001) from 2001 onwards compared with before 2001. No arterial punctures were performed after 2001 (0.0% vs 3.2%, P < 0.001). In Table 3, procedures that were followed by severe PR complications (fetal distress resulting in emergency CS or death) are compared with the remaining procedures by univariate analysis. Hydrops was not associated with severe PR complications (P = 0.315). Z‐hemoglobin (number of SDs from median concentration for gestational age), fetal paralysis, procedure access site and unsuccessful IUT were included in a multiple regression model. A positive association with severe PR complications was found for transamniotic (P = 0.030) and arterial (P < 0.001) transfusion sites, compared with transfusion into the fetal liver. There was a negative association of PR complications with fetal paralysis (P = 0.034). Intrahepatic and placental cord insertion sites were equally safe (P = 0.597). Table 3 Univariate analysis of characteristics of 34 intrauterine intravascular blood transfusions (IUTs) that were followed by severe procedure‐related (PR) complications, compared with 1644 remaining procedures in 589 fetuses with anemia caused by red‐cell alloimmunization Characteristic IUT with PR complication* (n = 34) Remaining IUTs (n = 1644) OR (95% CI) P Hydrops at IUT 7 (20.6) 231 (14.1)  1.6 (0.7–3.7) 0.315 GA at IUT (weeks) 31.1 (16.0 to 35.1) 29.9 (16.4 to 37.0) — 0.411

Table 3 Univariate analysis of characteristics of 34 intrauterine intravascular blood transfusions (IUTs) that were followed by severe procedure‐related (PR) complications, compared with 1644 remaining procedures in 589 fetuses with anemia caused by red‐cell alloimmunization Characteristic IUT with PR complication* (n = 34) Remaining IUTs (n = 1644) OR (95% CI) P Hydrops at IUT 7 (20.6) 231 (14.1)  1.6 (0.7–3.7) 0.315 GA at IUT (weeks) 31.1 (16.0 to 35.1) 29.9 (16.4 to 37.0) — 0.411 Z‐hemoglobin at IUT† −7.4 (−12.2 to −3.6) −6.8 (−11.7 to 1.4)  0.8 (0.7–1.0) 0.016 Fetal paralysis 23 (67.6) 1440 (87.6)  0.2 (0.1–0.5) 0.001 Procedure access site Liver 6 (17.6) 546 (33.2)  0.4 (0.2–1.0) 0.065 Placental cord insertion 11 (32.4) 787 (47.9)  0.5 (0.3–1.1) 0.083 Transamniotic ‘free loop’ 10 (29.4) 270 (16.4)  2.1 (1.0–4.5) 0.060 Artery 4 (11.8) 20 (1.2) 10.8 (3.5–33.6) 0.001 Intraperitoneal 0 (0) 13 (0.8) — 1.000 Other‡ 3 (8.8) 8 (0.5) 19.8 (5.0–78.2) 0.001 Unsuccessful IUT 3 (8.8) 30 (1.8)  5.2 (1.5–18.0) 0.027 Data are given as n (%) or median (range). * Procedures followed by fetal distress resulting in emergency Cesarean section within 24 h or fetal death. † Number of SDs from median concentration for gestational age (GA). ‡ Unknown vessel, heart, chorionic vein. OR, odds ratio.

Z‐hemoglobin at IUT† −7.4 (−12.2 to −3.6) −6.8 (−11.7 to 1.4)  0.8 (0.7–1.0) 0.016 Fetal paralysis 23 (67.6) 1440 (87.6)  0.2 (0.1–0.5) 0.001 Procedure access site Liver 6 (17.6) 546 (33.2)  0.4 (0.2–1.0) 0.065 Placental cord insertion 11 (32.4) 787 (47.9)  0.5 (0.3–1.1) 0.083 Transamniotic ‘free loop’ 10 (29.4) 270 (16.4)  2.1 (1.0–4.5) 0.060 Artery 4 (11.8) 20 (1.2) 10.8 (3.5–33.6) 0.001 Intraperitoneal 0 (0) 13 (0.8) — 1.000 Other‡ 3 (8.8) 8 (0.5) 19.8 (5.0–78.2) 0.001 Unsuccessful IUT 3 (8.8) 30 (1.8)  5.2 (1.5–18.0) 0.027 Data are given as n (%) or median (range). * Procedures followed by fetal distress resulting in emergency Cesarean section within 24 h or fetal death. † Number of SDs from median concentration for gestational age (GA). ‡ Unknown vessel, heart, chorionic vein. OR, odds ratio. DISCUSSION In this cohort of 589 fetuses treated with 1678 IUTs for fetal anemia caused by red‐cell immunization, we found an improvement in perinatal survival over time, from 88.6% in 1988–2000 to 97.0% from 2001 onwards. The incidence of PR complications decreased significantly in the last decade. PR loss rates declined from 4.7% to 1.8% per fetus and from 1.6% to 0.6% per procedure, a decrease that approached statistical significance. In spite of our policy not to apply routine antibiotics prior to transfusion, PR infection rate was extremely low (0.1% per procedure). These results suggest that intrauterine treatment for red‐cell immunization has become significantly safer in the past decade.

ure, a decrease that approached statistical significance. In spite of our policy not to apply routine antibiotics prior to transfusion, PR infection rate was extremely low (0.1% per procedure). These results suggest that intrauterine treatment for red‐cell immunization has become significantly safer in the past decade. Recently, three other European fetal therapy centers reported on PR complications and loss rates7, 8, 17. Pasman et al. found a comparable PR complication rate (1.5%) in 135 procedures performed between 2000 and 2014, although no perinatal death occurred in the 56 fetuses in their study8. We found lower overall PR perinatal loss rates compared with another recent European study, by Tiblad et al., in which four of 85 fetuses died as a direct result of the procedure (4.7% per fetus, 1.4% per procedure) performed between 1990 and 20107, and with the third study by Sainio et al., which had a 3.8% PR fetal‐loss rate17. None of these studies compared trends in results over time.

cent European study, by Tiblad et al., in which four of 85 fetuses died as a direct result of the procedure (4.7% per fetus, 1.4% per procedure) performed between 1990 and 20107, and with the third study by Sainio et al., which had a 3.8% PR fetal‐loss rate17. None of these studies compared trends in results over time. The increase in survival in our study may be explained, in part, by reduced severity of the disease at referral in the second cohort, reflected by a lower hydrops rate and higher hemoglobin concentrations at first IUT18, 19. This improvement reflects the optimization of the Dutch program for detection and prevention of red‐cell antibodies20, 21. We showed previously that fetal hydrops was associated with adverse outcome, both short‐ and long‐term18, 19, but not with occurrence of PR complications5, as confirmed in the present study. The decline in number of RhD immunizations during the study period is probably best explained by the introduction of routine prophylactic administration of anti‐D in the 30th week of gestation in 199822, in addition to postnatal anti‐D. It is likely that the most important factor associated with our low complication rates is the large number of IUTs performed annually at our center (mean, 62 per year vs 10, 14 and 38 per year in Tiblad et al.7, Pasman et al.8 and Sainio et al.17, respectively), which enhances operator and team experience and thus also diminishes PR complication rates6.

t factor associated with our low complication rates is the large number of IUTs performed annually at our center (mean, 62 per year vs 10, 14 and 38 per year in Tiblad et al.7, Pasman et al.8 and Sainio et al.17, respectively), which enhances operator and team experience and thus also diminishes PR complication rates6. We hypothesize that the extensive decline in PR complications is the result of avoiding possibly hazardous techniques in the more recent procedures. Risk factors for adverse outcome were identified previously5 and were confirmed in the current study as being arterial puncture, transamniotic ‘free loop’ needling and refraining from fetal paralysis. These technical aspects occurred significantly less frequently in procedures performed from 2001 onwards in our study, compared with procedures carried out before this timepoint. Furthermore, the fetal liver gained impressive popularity as a procedure access site, as this is associated with very low complication and loss rates and is considered to be a safe route of access5, 23, 24. In the previously mentioned studies with higher reported complication rates, 15.5%7 and 63.8%17 of transfusions were transamniotic.

etal liver gained impressive popularity as a procedure access site, as this is associated with very low complication and loss rates and is considered to be a safe route of access5, 23, 24. In the previously mentioned studies with higher reported complication rates, 15.5%7 and 63.8%17 of transfusions were transamniotic. Our current study shows that early IUT is still a hazardous procedure, as both NPR and PR complications occur more often before 20 weeks. Early IUT is technically more challenging, resulting in a higher complication risk. Unfortunately, evidence‐based studies on the benefit of intravenous immunoglobulin treatment to postpone the first IUT are still lacking. We are currently evaluating the effect of intravenous immunoglobulin in an international multicenter cohort study. One of the strengths of our study is that all complications were independently and thoroughly classified as PR or NPR. Furthermore, the size of our cohort is considerably larger than those in other published studies.

Our current study shows that early IUT is still a hazardous procedure, as both NPR and PR complications occur more often before 20 weeks. Early IUT is technically more challenging, resulting in a higher complication risk. Unfortunately, evidence‐based studies on the benefit of intravenous immunoglobulin treatment to postpone the first IUT are still lacking. We are currently evaluating the effect of intravenous immunoglobulin in an international multicenter cohort study. One of the strengths of our study is that all complications were independently and thoroughly classified as PR or NPR. Furthermore, the size of our cohort is considerably larger than those in other published studies. The retrospective design of this study carries some limitations. For example, the rationale for decisions on technical details of procedures by individual operators is difficult to determine retrospectively. However, because of the available evidence supporting transfusion techniques, randomization into different strategies to identify risk factors prospectively could be considered as unethical5, 8. Another limitation of this study could be that we did not address neonatal outcomes other than death. This was a deliberate choice, as our focus was on severe PR complications. Furthermore, short‐ and long‐term outcomes of IUTs have recently been thoroughly addressed by our group19.

ould be considered as unethical5, 8. Another limitation of this study could be that we did not address neonatal outcomes other than death. This was a deliberate choice, as our focus was on severe PR complications. Furthermore, short‐ and long‐term outcomes of IUTs have recently been thoroughly addressed by our group19. Our study demonstrates that IUTs should be a tailored treatment, with the chosen technique fine‐tuned to the patient's situation. We advocate that every operator should master all transfusion techniques and maintain experience by performing a sufficient number of transfusions per year in an experienced team. A minimum number of 10 transfusions annually for experienced operators has been suggested6. In order to achieve this target, centralization of fetal therapy is necessary. In summary, we found that IUT for red‐cell immunization has become a safer treatment option for fetal anemia. We believe that our current PR fetal‐loss rates (1.8% per fetus and 0.6% per procedure) can be considered ‘as good as it gets’ in experienced hands. For the future, we are focusing our research on non‐invasive treatment, such as immunomodulation with intravenous immunoglobulin.

safer treatment option for fetal anemia. We believe that our current PR fetal‐loss rates (1.8% per fetus and 0.6% per procedure) can be considered ‘as good as it gets’ in experienced hands. For the future, we are focusing our research on non‐invasive treatment, such as immunomodulation with intravenous immunoglobulin. ACKNOWLEDGMENTS We thank Annemieke Middeldorp and Monique Haak, Department of Obstetrics at Leiden University Medical Center, for their important contribution to our IUT team, Robertjan Meerman and Jenny Verdoes, Department of Obstetrics at Leiden University Medical Center, for their invaluable help retrieving the data and Ron Wolterbeek, Department of Medical Statistics and Bioinformatics at Leiden University, for his assistance with analyzing the data. This research was partly funded by a grant from Sanquin, Amsterdam, which did not influence design, conduct or publication of the study.

INTRODUCTION Pre‐eclampsia (PE) affects 2–5% of pregnancies1, 2, 3, 4, 5 and can result in intrauterine growth restriction (IUGR), renal or hepatic impairment, HELLP syndrome (hemolysis, elevated liver enzyme levels and low platelet count), eclampsia, and maternal and fetal mortality5, 6, 7. Early and late manifestations of PE differ in time of onset of symptoms, relative frequency, placental morphology, genetic risk and risk of adverse outcomes8, 9, 10, 11, 12, 13. Early‐onset PE is associated with a higher incidence of adverse perinatal outcomes, including oligohydramnios, Apgar score < 7, stillbirth and early neonatal death, compared with late‐onset PE14, 15. As early intervention is important to improve maternal and fetal outcomes16 and the classical clinical markers of PE (hypertension and proteinuria) are poorly predictive of those who will develop the condition, markers of angiogenesis have been examined as aids to PE prediction.

eath, compared with late‐onset PE14, 15. As early intervention is important to improve maternal and fetal outcomes16 and the classical clinical markers of PE (hypertension and proteinuria) are poorly predictive of those who will develop the condition, markers of angiogenesis have been examined as aids to PE prediction. A key feature of PE is placental insufficiency. Dysregulation of pro‐ and antiangiogenic factors is thought to be causally linked to the condition7, 17, 18; before and during PE, maternal serum concentrations of antiangiogenic soluble fms‐like tyrosine kinase‐1 (sFlt‐1) are increased and levels of proangiogenic placental growth factor (PlGF) are decreased19, 20. A high sFlt‐1/PlGF ratio has been linked with PE and demonstrated before clinical onset of the condition, and differences in sFlt‐1 and PlGF have been observed between early‐ and late‐onset PE21, 22, 23, 24, 25, 26, 27, 28, 29. The Elecsys® immunoassay sFlt‐1/PlGF ratio is CE‐IVD (Conformité Européenne–In Vitro Diagnostics) approved as a diagnostic aid for PE with gestational age‐specific cut‐off values, and as an aid in short‐term prediction of PE in women with suspected PE26, 30, 31. The Prediction of Short‐Term Outcome in Pregnant Women with Suspected PE Study (PROGNOSIS) developed a cut‐off‐based PE prediction model. Optimum sFlt‐1/PlGF ratio cut‐off levels of ≤ 38 and > 38 were identified to rule out and rule in, respectively, PE, in women with singleton pregnancy at 24 + 0 to 36 + 6 weeks' gestation32. However, the predictive value of the sFlt‐1/PlGF ratio has not been examined specifically for early‐onset PE.

E prediction model. Optimum sFlt‐1/PlGF ratio cut‐off levels of ≤ 38 and > 38 were identified to rule out and rule in, respectively, PE, in women with singleton pregnancy at 24 + 0 to 36 + 6 weeks' gestation32. However, the predictive value of the sFlt‐1/PlGF ratio has not been examined specifically for early‐onset PE. This study, the Study of Early Pre‐eclampsia in Spain (STEPS), aimed to evaluate the sFlt‐1/PlGF ratio at 20, 24 and 28 weeks as a predictive marker for early‐onset PE in women at risk of PE.

E prediction model. Optimum sFlt‐1/PlGF ratio cut‐off levels of ≤ 38 and > 38 were identified to rule out and rule in, respectively, PE, in women with singleton pregnancy at 24 + 0 to 36 + 6 weeks' gestation32. However, the predictive value of the sFlt‐1/PlGF ratio has not been examined specifically for early‐onset PE. This study, the Study of Early Pre‐eclampsia in Spain (STEPS), aimed to evaluate the sFlt‐1/PlGF ratio at 20, 24 and 28 weeks as a predictive marker for early‐onset PE in women at risk of PE. METHODS Study design and participants STEPS was a prospective, double‐blind, multicenter (10 study sites in Spain) study, performed between October 2010 and March 2013, and enrolled pregnant women at risk of PE. To be considered at risk of PE, women had to meet one of the following inclusion criteria: PE, eclampsia, HELLP syndrome or IUGR in a previous pregnancy; pre‐existing chronic hypertension without proteinuria; gestational hypertension (new‐onset hypertension in pregnancy); pre‐existing renal disease (kidney transplantation or creatinine clearance < 60 mL/min); pre‐existing diabetes mellitus Type I (insulin dependent); mean uterine artery Doppler pulsatility index (UtA‐PI) > 1.45 (at 19–20 weeks); thrombophilia (antiphospholipid syndrome, protein C deficiency, protein S deficiency, antithrombin deficiency, factor V Leiden mutation); multiple pregnancy; age ≥ 40 years and conceived with assisted reproductive technologies (ART). Women with two or more of the following risk factors were also included: nulliparity; body mass index ≥ 35 kg/m2; diastolic blood pressure > 80 mmHg at study inclusion; age ≥ 40 years; and family history (mother or sister) of PE, eclampsia or HELLP syndrome. Women were excluded if they were both hypertensive and had proteinuria or if major fetal malformations/chromosome disorders were observed.

iparity; body mass index ≥ 35 kg/m2; diastolic blood pressure > 80 mmHg at study inclusion; age ≥ 40 years; and family history (mother or sister) of PE, eclampsia or HELLP syndrome. Women were excluded if they were both hypertensive and had proteinuria or if major fetal malformations/chromosome disorders were observed. Women provided informed, signed consent. The protocol was approved by applicable national/regional independent ethics committees and institutional review boards (Table S1). The study adhered to the Guidelines for Good Clinical Practice. The primary objective of the study was to demonstrate that the sFlt‐1/PlGF ratio was a predictive marker for early‐onset PE. Secondary objectives included evaluation of sFlt‐1/PlGF ratio as a predictor of late‐onset PE and the use of the sFlt‐1/PlGF ratio for differentiation of hypertension from PE. Diagnostic criteria For consistency, investigators used predefined diagnostic criteria (Table 1) based on the Report of the National High Blood Pressure Education Program Working Group on High Blood Pressure in Pregnancy33. PE was defined as newly occurring hypertension (systolic blood pressure ≥ 140 mmHg and/or diastolic blood pressure ≥ 90 mmHg) with newly occurring proteinuria after 20 weeks. To be considered early onset, PE had to occur before 34 + 0 weeks. Table 1 Diagnostic criteria in Study of Early Pre‐eclampsia in Spain (STEPS)

Diagnostic criteria For consistency, investigators used predefined diagnostic criteria (Table 1) based on the Report of the National High Blood Pressure Education Program Working Group on High Blood Pressure in Pregnancy33. PE was defined as newly occurring hypertension (systolic blood pressure ≥ 140 mmHg and/or diastolic blood pressure ≥ 90 mmHg) with newly occurring proteinuria after 20 weeks. To be considered early onset, PE had to occur before 34 + 0 weeks. Table 1 Diagnostic criteria in Study of Early Pre‐eclampsia in Spain (STEPS) Diagnosis Criteria Hypertension Systolic blood pressure ≥ 140 mmHg and/or diastolic blood pressure ≥ 90 mmHg (on two occasions at least 6 h apart) Chronic hypertension Hypertension (systolic blood pressure ≥ 140 mmHg and/or diastolic blood pressure ≥ 90 mmHg) diagnosed before pregnancy or in first half of pregnancy (< 20 weeks) and continued for > 12 weeks after delivery Proteinuria For determination of urinary protein using test strips, a value of 1+ was not considered reliable for diagnosis of PE.

stolic blood pressure ≥ 140 mmHg and/or diastolic blood pressure ≥ 90 mmHg) diagnosed before pregnancy or in first half of pregnancy (< 20 weeks) and continued for > 12 weeks after delivery Proteinuria For determination of urinary protein using test strips, a value of 1+ was not considered reliable for diagnosis of PE. Data were reconfirmed with protein test on 24‐h urine (≥ 0.3 g protein/24 h); in an emergency, if it was not possible to determine protein in 24‐h urine, protein determination was carried out on isolated urine sample (≥ 30 mg protein/dL or protein/creatinine ratio ≥ 30 mg protein/mmol creatinine) Gestational hypertension New‐onset hypertension (systolic blood pressure ≥ 140 mmHg and/or diastolic blood pressure ≥ 90 mmHg) after 20 weeks of pregnancy, which resolved by 12 weeks postpartum PE New‐onset hypertension (systolic blood pressure ≥ 140 mmHg and/or diastolic blood pressure ≥ 90 mmHg) and new‐onset proteinuria after 20 weeks of pregnancy Severe PE PE plus one or more of the following: systolic blood pressure ≥ 160 mmHg and/or diastolic blood pressure ≥ 110 mmHg (on two occasions at least 6 h apart); proteinuria (>5 g protein/24 h or test strip ≥ 3+ in two urine samples collected at random at least 4 h apart); impairment of renal function (serum creatinine ≥ 1.2 mg/dL unless known to be elevated previously or oliguria < 500 mL/24 h); pulmonary edema; impairment of hepatic function (elevated liver enzymes, epigastric pain or right upper quadrant pain caused by distension of Glisson's capsule); neurological symptoms (cerebral or visual disturbances, severe headache); hematological disturbances (thrombocytopenia, hemolysis); IUGR Eclampsia New‐onset tonic–clonic convulsions in women with PE, not attributable to any other cause Early‐ and late‐onset PE Early onset: PE developing < 34 + 0 weeks; late onset: PE developing ≥ 34 + 0 weeks HELLP syndrome Increased ASAT (> 70 IU/L); decreased platelet count (< 100 000/μL); increased LDH (> 600 IU/L) IUGR Estimated fetal weight or abdominal circumference < 10th percentile (adjusted for gender/race in accordance with tables normally used by study center). Presence of pathological process that inhibits expression of normal intrinsic growth potential.

decreased platelet count (< 100 000/μL); increased LDH (> 600 IU/L) IUGR Estimated fetal weight or abdominal circumference < 10th percentile (adjusted for gender/race in accordance with tables normally used by study center). Presence of pathological process that inhibits expression of normal intrinsic growth potential. Pathological process must be demonstrated at least once after 22 weeks, according to either oligohydramnios (amniotic fluid index < 10th percentile) or pathological flow in umbilical artery (pulsatility index > 95th percentile) SGA neonate Estimated fetal weight or abdominal circumference < 10th percentile (adjusted for gender/race in accordance with tables normally used by study center); no pathological process Preterm birth Delivery before end of 37 weeks (e.g. gestational age of 36 + 6 weeks would be recorded as 36 completed weeks of pregnancy and baby would be defined as preterm) ASAT, aspartate aminotransferase; HELLP, hemolysis, elevated liver enzymes and low platelet count; IUGR, intrauterine growth restriction; LDH, lactate dehydrogenase; PE, pre‐eclampsia; SGA, small‐for‐gestational‐age.

36 + 6 weeks would be recorded as 36 completed weeks of pregnancy and baby would be defined as preterm) ASAT, aspartate aminotransferase; HELLP, hemolysis, elevated liver enzymes and low platelet count; IUGR, intrauterine growth restriction; LDH, lactate dehydrogenase; PE, pre‐eclampsia; SGA, small‐for‐gestational‐age. Data collection and visits At gestational weeks 19–20 (Visit 1), 23–24 (Visit 2) and 27–28 (Visit 3), participants underwent a blood test to determine the sFlt‐1/PlGF ratio, Doppler examination of the uterine arteries and assessment of blood pressure (measured by validated automated devices), proteinuria, PE status, hemoglobin, platelets and uric acid levels. Postpartum, additional data were collected, including blood pressure, type of delivery, Apgar score, weight of placenta, neonatal outcomes (perinatal/fetal death, delivery < 34 weeks, IUGR, placental abruption, respiratory distress syndrome, necrotizing enterocolitis, intraventricular hemorrhage) and maternal outcomes (maternal death, pulmonary edema, acute renal failure, cerebral hemorrhage, cerebral thrombosis, disseminated intravascular coagulation). Unplanned visits could be carried out in the event of complications. Serum samples (≥ 2 mL) were collected according to a standard operating procedure and were analyzed at the individual study sites. Results were checked for consistency between study sites by central analysis at Hospital Universitario Central de Asturias.

Data collection and visits At gestational weeks 19–20 (Visit 1), 23–24 (Visit 2) and 27–28 (Visit 3), participants underwent a blood test to determine the sFlt‐1/PlGF ratio, Doppler examination of the uterine arteries and assessment of blood pressure (measured by validated automated devices), proteinuria, PE status, hemoglobin, platelets and uric acid levels. Postpartum, additional data were collected, including blood pressure, type of delivery, Apgar score, weight of placenta, neonatal outcomes (perinatal/fetal death, delivery < 34 weeks, IUGR, placental abruption, respiratory distress syndrome, necrotizing enterocolitis, intraventricular hemorrhage) and maternal outcomes (maternal death, pulmonary edema, acute renal failure, cerebral hemorrhage, cerebral thrombosis, disseminated intravascular coagulation). Unplanned visits could be carried out in the event of complications. Serum samples (≥ 2 mL) were collected according to a standard operating procedure and were analyzed at the individual study sites. Results were checked for consistency between study sites by central analysis at Hospital Universitario Central de Asturias. Maternal serum levels of sFlt‐1 and PlGF were determined using the fully automated Elecsys sFlt‐1 and Elecsys PlGF assays on the cobas® e electrochemiluminescence immunoassay platform (Roche Diagnostics GmbH, Mannheim, Germany) and the sFlt‐1/PlGF ratio was calculated27, 30, 31. The sFlt‐1/PlGF ratio results were concealed from both patients and carers to ensure that they did not affect the clinical monitoring of patients.

sys PlGF assays on the cobas® e electrochemiluminescence immunoassay platform (Roche Diagnostics GmbH, Mannheim, Germany) and the sFlt‐1/PlGF ratio was calculated27, 30, 31. The sFlt‐1/PlGF ratio results were concealed from both patients and carers to ensure that they did not affect the clinical monitoring of patients. Adverse events were recorded, although the study was non‐interventional. Statistical analysis To obtain 100 cases of PE, it was calculated that 800 pregnant women would need to be included in the study, based on a presumed prevalence of PE of 12% (including both singleton and multiple pregnancies). The sFlt‐1/PlGF ratio was log‐transformed to correct for right skewness prior to any calculation. Differences in means between independent groups were assessed using analysis of variance (ANOVA) or Student's t‐test in the case of homogeneity of variances, and using generalized least squares in the case of heteroscedasticity. Appropriateness of the methods was assessed by evaluation of the plots of residuals.

ation. Differences in means between independent groups were assessed using analysis of variance (ANOVA) or Student's t‐test in the case of homogeneity of variances, and using generalized least squares in the case of heteroscedasticity. Appropriateness of the methods was assessed by evaluation of the plots of residuals. To develop a predictive model of PE, multivariate logistic regression was used considering maternal characteristics, medical history and biomarkers as potential predictors. The variables that were finally included in the early‐PE prediction model were selected according to the results of a logistic regression model with L1 penalization (‘lasso’ technique)34. The coefficients derived from the multivariate analysis were used as weights in a nomogram to predict early PE. Performances of the models were evaluated by receiver–operating characteristics (ROC) curves and areas under the curve (AUC) with 95% CIs. All statistical analyses were performed using R (version 3.1.2) and R‐packages rms (version 4.2‐1) and ROCR (version 1.0‐5).

To develop a predictive model of PE, multivariate logistic regression was used considering maternal characteristics, medical history and biomarkers as potential predictors. The variables that were finally included in the early‐PE prediction model were selected according to the results of a logistic regression model with L1 penalization (‘lasso’ technique)34. The coefficients derived from the multivariate analysis were used as weights in a nomogram to predict early PE. Performances of the models were evaluated by receiver–operating characteristics (ROC) curves and areas under the curve (AUC) with 95% CIs. All statistical analyses were performed using R (version 3.1.2) and R‐packages rms (version 4.2‐1) and ROCR (version 1.0‐5). RESULTS Study participants Overall, 729 women were eligible for analysis, including 447 with singleton pregnancy and 282 with multiple pregnancy (twin pregnancy, n = 276; triplet pregnancy, n = 6). A total of 78 (10.7%) women developed PE (singleton pregnancy, n = 42; multiple pregnancy, n = 36), of which 24 were early‐onset PE (singleton pregnancy, n = 14; multiple pregnancy, n = 10) and 54 were late‐onset PE (singleton pregnancy, n = 28; multiple pregnancy, n = 26) (Figure 1). The number of participants per study site is reported in Table S2. Women who developed early‐onset PE had higher systolic and diastolic blood pressures, mean arterial blood pressure (MAP) and lower gestational age at delivery compared with the control group (women who did not develop PE/hypertension during the entire pregnancy) (Table 2).

ipants per study site is reported in Table S2. Women who developed early‐onset PE had higher systolic and diastolic blood pressures, mean arterial blood pressure (MAP) and lower gestational age at delivery compared with the control group (women who did not develop PE/hypertension during the entire pregnancy) (Table 2). Figure 1 Flowchart of participants in Study of Early Pre‐eclampsia in Spain (STEPS). *Reasons for exclusion: inclusion criteria not met (n = 4); signed consent given but did not start study (n = 28); miscarriage (n = 7); termination of pregnancy due to fetal malformations (n = 8); lost to follow‐up (n = 13); placental abruption at 26 weeks (n = 1); completed follow‐up until 28 weeks but data could not be retrieved because of delivery in another setting (n = 29). IUGR, intrauterine growth restriction; M, multiple pregnancy; PE, pre‐eclampsia; S, singleton pregnancy. UOG-17373-FIG-0001-bTable 2 Baseline characteristics of women who developed early‐ or late‐onset pre‐eclampsia (PE) and those who did not develop PE (controls) Characteristic Controls (n = 651) Early‐onset PE (n = 24) Late‐onset PE (n = 54) Age (years) 34.6 ± 5.3  35.6 ± 3.9 34.7 ± 0.7 Body mass index (kg/m2) 26.7 ± 6.0  28.5 ± 6.4 27.9 ± 7.3 Systolic blood pressure (mmHg) 119.1 ± 13.7  127.9 ± 13.5* 125.0 ± 16.0* Diastolic blood pressure (mmHg) 73.7 ± 11.3 77.5 ± 8.5* 77.7 ± 12.0* Mean arterial pressure (mmHg) 88.9 ± 11.1 94.3 ± 8.1* 93.4 ± 12.2*

Characteristic Controls (n = 651) Early‐onset PE (n = 24) Late‐onset PE (n = 54) Age (years) 34.6 ± 5.3  35.6 ± 3.9 34.7 ± 0.7 Body mass index (kg/m2) 26.7 ± 6.0  28.5 ± 6.4 27.9 ± 7.3 Systolic blood pressure (mmHg) 119.1 ± 13.7  127.9 ± 13.5* 125.0 ± 16.0* Diastolic blood pressure (mmHg) 73.7 ± 11.3 77.5 ± 8.5* 77.7 ± 12.0* Mean arterial pressure (mmHg) 88.9 ± 11.1 94.3 ± 8.1* 93.4 ± 12.2* Multiple pregnancy 246 (37.8) 10 (41.7) 26 (48.1) Gestational age at delivery (weeks) 37.5 ± 2.7  31.8 ± 3.5* 36.6 ± 1.4 Birth weight of first infant (g)  2911 ± 721 (n = 646) 1745 ± 830 (n = 22)* 2759 ± 584 (n = 54) Birth weight of second infant (g) 2303 ± 552 (n = 243) 1807 ± 378 (n = 9)† 2285 ± 364 (n = 26) Birth weight of third infant (g) 1221 ± 689 (n = 4) 1445 ± 304 (n = 2) — (n = 0) Nulliparous 272 (41.8) 15 (62.5) 31 (57.4) Previous PE 101 (15.5) 9 (37.5)* 14 (25.9) Family history of PE 23 (3.5) 1 (4.2) 5 (9.3) Previous IUGR 55 (8.4) 3 (12.5) 5 (9.3) Chronic hypertension  81 (12.4) 6 (25.0) 12 (22.2) Gestational hypertension  4 (0.6) 0 (0) 3 (5.6) Nephropathy  6 (0.9) 0 (0) 1 (1.9) Diabetes mellitus Type 1  42 (6.5)  1 (4.2) 1 (1.9) Thrombophilia  50 (7.7)  1 (4.2) 3 (5.6) Conceived by assisted reproduction  93 (14.3) 5 (20.8) 11 (20.4) Smoker at enrollment  80 (12.3) 1 (4.2) 3 (5.6) Abnormal UtA Doppler  8 (1.2) 1 (4.2) 2 (3.7) Data are given as mean ± SD or n (%). PE groups compared with controls using Dunnett's test: * P < 0.001; † P < 0.05, after adjustment by Bonferroni correction. IUGR, intrauterine growth restriction; UtA, uterine artery.

Multiple pregnancy 246 (37.8) 10 (41.7) 26 (48.1) Gestational age at delivery (weeks) 37.5 ± 2.7  31.8 ± 3.5* 36.6 ± 1.4 Birth weight of first infant (g)  2911 ± 721 (n = 646) 1745 ± 830 (n = 22)* 2759 ± 584 (n = 54) Birth weight of second infant (g) 2303 ± 552 (n = 243) 1807 ± 378 (n = 9)† 2285 ± 364 (n = 26) Birth weight of third infant (g) 1221 ± 689 (n = 4) 1445 ± 304 (n = 2) — (n = 0) Nulliparous 272 (41.8) 15 (62.5) 31 (57.4) Previous PE 101 (15.5) 9 (37.5)* 14 (25.9) Family history of PE 23 (3.5) 1 (4.2) 5 (9.3) Previous IUGR 55 (8.4) 3 (12.5) 5 (9.3) Chronic hypertension  81 (12.4) 6 (25.0) 12 (22.2) Gestational hypertension  4 (0.6) 0 (0) 3 (5.6) Nephropathy  6 (0.9) 0 (0) 1 (1.9) Diabetes mellitus Type 1  42 (6.5)  1 (4.2) 1 (1.9) Thrombophilia  50 (7.7)  1 (4.2) 3 (5.6) Conceived by assisted reproduction  93 (14.3) 5 (20.8) 11 (20.4) Smoker at enrollment  80 (12.3) 1 (4.2) 3 (5.6) Abnormal UtA Doppler  8 (1.2) 1 (4.2) 2 (3.7) Data are given as mean ± SD or n (%). PE groups compared with controls using Dunnett's test: * P < 0.001; † P < 0.05, after adjustment by Bonferroni correction. IUGR, intrauterine growth restriction; UtA, uterine artery. sFlt‐1, PlGF and sFlt‐1/PlGF ratio measurements In the control group, median sFlt‐1/PlGF ratio remained low (< 7) between 20 and 28 weeks' gestation (Table 3). In women who developed early‐onset PE, median sFlt‐1/PlGF ratio was already higher (14.5) at 20 weeks' gestation and increased further to 18.4 at 24 weeks and 51.9 at 28 weeks. There was little change in the median sFlt‐1/PlGF ratio between 20 and 28 weeks in women who developed late‐onset PE, remaining low throughout at < 7.

n who developed early‐onset PE, median sFlt‐1/PlGF ratio was already higher (14.5) at 20 weeks' gestation and increased further to 18.4 at 24 weeks and 51.9 at 28 weeks. There was little change in the median sFlt‐1/PlGF ratio between 20 and 28 weeks in women who developed late‐onset PE, remaining low throughout at < 7. Table 3 Measurements of soluble fms‐like tyrosine kinase‐1 (sFlt‐1), placental growth factor (PlGF) and sFlt‐1/PlGF ratio in maternal serum at 20, 24 and 28 weeks in women who developed early‐ or late‐onset pre‐eclampsia (PE) and in those who did not develop PE (controls) Biomarker Controls Early‐onset PE Late‐onset PE 20 weeks n 612 21 52 PlGF (pg/mL) 264.5 (172.0–403.6) 193.1 (68.0–262.4) 267.8 (151.5–414.0) sFlt‐1 (pg/mL)  1623.0 (1081.0–2531.0)   1972.0 (1331.0–3473.0)  1967.0 (1120.5–2903.8) sFlt‐1/PlGF ratio 6.3 (4.1–9.3)   14.5 (5.5–43.7)  6.7 (4.6–9.9) 24 weeks n 580 20 52 PlGF (pg/mL) 424.5 (277.0–615.6) 168.9 (62.1–329.7) 415.0 (259.7–595.7) sFlt‐1 (pg/mL)  1725.0 (1123.5–2674.3)   3127.5 (1961.8–4202.5)  1882.5 (1134.5–3115.8) sFlt‐1/PlGF ratio 4.0 (2.6–6.3)   18.4 (8.2–57.9)  4.7 (2.8–7.2) 28 weeks n 557 16 49 PlGF (pg/mL) 540.0 (339.0–821.5) 176.5 (67.2–278.6) 335.0 (263.0–485.9) sFlt‐1 (pg/mL)  1826.0 (1231.0–2766.0)    6370.0 (2385.3–8788.3)   2499.0 (1522.0–3681.0) sFlt‐1/PlGF ratio 3.3 (2.0–5.9)    51.9 (11.5–145.6) 6.0 (3.8–10.5) Data are given as median (interquartile range) unless stated otherwise.

n 580 20 52 PlGF (pg/mL) 424.5 (277.0–615.6) 168.9 (62.1–329.7) 415.0 (259.7–595.7) sFlt‐1 (pg/mL)  1725.0 (1123.5–2674.3)   3127.5 (1961.8–4202.5)  1882.5 (1134.5–3115.8) sFlt‐1/PlGF ratio 4.0 (2.6–6.3)   18.4 (8.2–57.9)  4.7 (2.8–7.2) 28 weeks n 557 16 49 PlGF (pg/mL) 540.0 (339.0–821.5) 176.5 (67.2–278.6) 335.0 (263.0–485.9) sFlt‐1 (pg/mL)  1826.0 (1231.0–2766.0)    6370.0 (2385.3–8788.3)   2499.0 (1522.0–3681.0) sFlt‐1/PlGF ratio 3.3 (2.0–5.9)    51.9 (11.5–145.6) 6.0 (3.8–10.5) Data are given as median (interquartile range) unless stated otherwise. Mean sFlt‐1 levels and PlGF levels were significantly different between singleton and multiple pregnancies at 20, 24 and 28 weeks (P = 0.001). However, the mean sFlt‐1/PlGF ratio was only significantly different at 28 weeks' gestation (P = 0.001) (Table S3) and, at all timepoints, the difference between median values in singleton and multiple pregnancies was small. sFlt‐1/PlGF ratio: prediction of PE Compared with control participants, the sFlt‐1/PlGF ratio was consistently significantly higher in women with early‐onset PE (P < 0.001 at all timepoints) (Figure 2). Women with early‐onset PE also had significantly higher sFlt‐1/PlGF ratios at 20, 24 and 28 weeks relative to women with chronic or gestational hypertension and women with late‐onset PE (Figure 3). Differences between early‐onset PE and control/hypertension/late‐onset PE became more pronounced as the pregnancy progressed.

en with early‐onset PE also had significantly higher sFlt‐1/PlGF ratios at 20, 24 and 28 weeks relative to women with chronic or gestational hypertension and women with late‐onset PE (Figure 3). Differences between early‐onset PE and control/hypertension/late‐onset PE became more pronounced as the pregnancy progressed. Figure 2 Soluble fms‐like tyrosine kinase‐1 (sFlt‐1)/placental growth factor (PlGF) ratio at 20, 24 and 28 weeks in control group of women who did not develop pre‐eclampsia (PE; ) and in those who developed early‐onset () or late‐onset () PE. Comparison with controls: *P < 0.001; †P = 0.15; ‡P = 0.21. UOG-17373-FIG-0002-bFigure 3 Box‐and‐whisker plots of soluble fms‐like tyrosine kinase‐1 (sFlt‐1)/placental growth factor (PlGF) ratio in control women who did not develop pre‐eclampsia (PE) and in those who developed chronic hypertension (CH), gestational hypertension (GH), late‐onset PE or early‐onset PE at: (a) 20 weeks, (b) 24 weeks and (c) 28 weeks. †P < 0.001 in comparison with early‐onset PE. Boxes with internal lines represent median and interquartile range, whiskers are 1.5 × interquartile range and stars are outliers.

oped chronic hypertension (CH), gestational hypertension (GH), late‐onset PE or early‐onset PE at: (a) 20 weeks, (b) 24 weeks and (c) 28 weeks. †P < 0.001 in comparison with early‐onset PE. Boxes with internal lines represent median and interquartile range, whiskers are 1.5 × interquartile range and stars are outliers. UOG-17373-FIG-0003-bWomen with late‐onset PE were not easily differentiated from control participants by the sFlt‐1/PlGF ratio at 20 and 24 weeks (difference was non‐significant at 20 (P = 0.15) and 24 (P = 0.21) weeks, Figure 2). At 28 weeks, there was a statistically significant difference in the sFlt‐1/PlGF ratio between women with late‐onset PE and control participants (P < 0.001), although the numerical difference in the median ratio was small (2.7) (Table 3).

fference was non‐significant at 20 (P = 0.15) and 24 (P = 0.21) weeks, Figure 2). At 28 weeks, there was a statistically significant difference in the sFlt‐1/PlGF ratio between women with late‐onset PE and control participants (P < 0.001), although the numerical difference in the median ratio was small (2.7) (Table 3). Development of a prediction model for early‐onset PE Prediction models for early‐onset PE were developed, which included variations of the following factors: sFlt‐1/PlGF ratio, PlGF, UtA‐PI, MAP, being parous, previous PE and use of ART. The AUC was optimal for a model including the sFlt‐1/PlGF ratio, MAP, being parous and previous PE (hereafter referred to as the ‘early‐onset PE prediction model’) (Figure S1) compared with models that used the sFlt‐1/PlGF ratio alone or UtA‐PI alone (Table 4). The accuracy of the prediction model was not substantially improved by including ART or UtA‐PI and ART in the model. At 20 and 24 weeks, including these two parameters in the model increased the AUC from 0.86 (95% CI, 0.77–0.95) to 0.87 (95% CI, 0.79–0.96), and from 0.91 (95% CI, 0.85–0.97) to 0.92 (95% CI, 0.85–0.97), respectively. However, at 28 weeks, including UtA‐PI and ART in the model reduced the AUC from 0.93 (95% CI, 0.86–0.99) to 0.91 (95% CI, 0.82–0.99) (Figure 4). A nomogram for prediction risk is presented in Figures S2 and S3. The detection rate for early‐onset PE using different prediction models is reported in Table 5.

espectively. However, at 28 weeks, including UtA‐PI and ART in the model reduced the AUC from 0.93 (95% CI, 0.86–0.99) to 0.91 (95% CI, 0.82–0.99) (Figure 4). A nomogram for prediction risk is presented in Figures S2 and S3. The detection rate for early‐onset PE using different prediction models is reported in Table 5. Table 4 Prediction of early‐onset pre‐eclampsia (PE) at 20, 24 and 28 weeks using different individual parameters and early‐onset PE prediction model Prediction parameter AUC (95% CI) 20 weeks Early‐onset PE prediction model 0.86 (0.77–0.95) MAP 0.67 (0.55–0.79) UtA‐PI 0.50 (0.35–0.66) PlGF 0.70 (0.58–0.82) sFlt‐1 0.61 (0.49–0.74) sFlt‐1/PlGF ratio 0.77 (0.65–0.89) 24 weeks Early‐onset PE prediction model 0.91 (0.85–0.97) MAP 0.72 (0.62–0.83) UtA‐PI 0.55 (0.39–0.72) PlGF 0.81 (0.72–0.90) sFlt‐1 0.71 (0.58–0.84) sFlt‐1/PlGF ratio 0.86 (0.76–0.96) 28 weeks Early‐onset PE prediction model 0.93 (0.86–0.99) MAP 0.77 (0.66–0.89) UtA‐PI 0.63 (0.45–0.80) PlGF 0.86 (0.78–0.94) sFlt‐1 0.81 (0.67–0.95) sFlt‐1/PlGF ratio 0.89 (0.79–0.98) Early‐onset PE prediction model includes soluble fms‐like tyrosine kinase 1(sFlt‐1)/placental growth factor (PlGF) ratio, mean arterial pressure (MAP), being parous and previous PE. AUC, area under the receiver–operating characteristics curve; UtA‐PI, uterine artery pulsatility index.

Prediction parameter AUC (95% CI) 20 weeks Early‐onset PE prediction model 0.86 (0.77–0.95) MAP 0.67 (0.55–0.79) UtA‐PI 0.50 (0.35–0.66) PlGF 0.70 (0.58–0.82) sFlt‐1 0.61 (0.49–0.74) sFlt‐1/PlGF ratio 0.77 (0.65–0.89) 24 weeks Early‐onset PE prediction model 0.91 (0.85–0.97) MAP 0.72 (0.62–0.83) UtA‐PI 0.55 (0.39–0.72) PlGF 0.81 (0.72–0.90) sFlt‐1 0.71 (0.58–0.84) sFlt‐1/PlGF ratio 0.86 (0.76–0.96) 28 weeks Early‐onset PE prediction model 0.93 (0.86–0.99) MAP 0.77 (0.66–0.89) UtA‐PI 0.63 (0.45–0.80) PlGF 0.86 (0.78–0.94) sFlt‐1 0.81 (0.67–0.95) sFlt‐1/PlGF ratio 0.89 (0.79–0.98) Early‐onset PE prediction model includes soluble fms‐like tyrosine kinase 1(sFlt‐1)/placental growth factor (PlGF) ratio, mean arterial pressure (MAP), being parous and previous PE. AUC, area under the receiver–operating characteristics curve; UtA‐PI, uterine artery pulsatility index. Figure 4 Receiver–operating characteristcs curves for prediction of early‐onset pre‐eclampsia (PE) using different models at 20 weeks (a), 24 weeks (b) and 28 weeks (c). Tables S4 and 5 present numerical values for areas under curves. Early‐onset PE prediction model () includes soluble fms‐like tyrosine kinase‐1 (sFlt‐1)/placental growth factor (PlGF) ratio, mean arterial pressure (MAP), being parous and previous PE. ART, assisted reproductive technologies; UtA‐PI, uterine artery pulsatility index. , sFlt‐1/PlGF ratio, MAP, being parous, previous PE, ART. , sFlt‐1/PlGF ratio, MAP, being parous, previous PE, UtA‐PI, ART. , MAP, being parous, previous PE, ART, PlGF. , MAP, being parous, previous PE, UtA‐PI, ART, PlGF.

nd previous PE. ART, assisted reproductive technologies; UtA‐PI, uterine artery pulsatility index. , sFlt‐1/PlGF ratio, MAP, being parous, previous PE, ART. , sFlt‐1/PlGF ratio, MAP, being parous, previous PE, UtA‐PI, ART. , MAP, being parous, previous PE, ART, PlGF. , MAP, being parous, previous PE, UtA‐PI, ART, PlGF. UOG-17373-FIG-0004-bTable 5 Prediction rates of early‐onset pre‐eclampsia (PE) using different models at 20, 24 and 28 weeks Detection rate (%) Prediction model FPR = 5% FPR = 10% 20 weeks Early‐onset PE prediction model 45 60 sFlt‐1/PlGF ratio, MAP, being parous, previous PE, UtA‐PI and ART 50 60 sFlt‐1/PlGF ratio, MAP, being parous, previous PE and ART 55 60 MAP, being parous, previous PE, UtA‐PI, ART and PlGF 35 55 MAP, being parous, previous PE, ART and PlGF 45 45 24 weeks Early‐onset PE prediction model 60 70 sFlt‐1/PlGF ratio, MAP, being parous, previous PE, UtA‐PI and ART 72 78 sFlt‐1/PlGF ratio, MAP, being parous, previous PE and ART 60 70 MAP, being parous, previous PE, UtA‐PI, ART and PlGF 56 67 MAP, being parous, previous PE, ART and PlGF 56 67 28 weeks Early‐onset PE prediction model 81 81 sFlt‐1/PlGF ratio, MAP, being parous, previous PE, UtA‐PI and ART 73 80 sFlt‐1/PlGF ratio, MAP, being parous, previous PE and ART 81 81 MAP, being parous, previous PE, UtA‐PI, ART and PlGF 53 53 MAP, being parous, previous PE, ART and PlGF 53 53 Early‐onset PE prediction model includes soluble fms‐like tyrosine kinase 1(sFlt‐1)/placental growth factor (PlGF) ratio, mean arterial pressure (MAP), being parous and previous PE.

s, previous PE and ART 81 81 MAP, being parous, previous PE, UtA‐PI, ART and PlGF 53 53 MAP, being parous, previous PE, ART and PlGF 53 53 Early‐onset PE prediction model includes soluble fms‐like tyrosine kinase 1(sFlt‐1)/placental growth factor (PlGF) ratio, mean arterial pressure (MAP), being parous and previous PE. Areas under receiver–operating chracteristics curves for each model are provided in Table S4. ART, assisted reproductive technologies; FPR, false‐positive rate; UtA‐PI, uterine artery pulsatility index. We also compared the performance of a standard prediction model (maternal history, MAP and UtA‐PI) with the same model but including the sFlt‐1/PlGF immunoassay ratio to estimate early‐onset PE risk at 20, 24 and 28 weeks' gestation. The addition of the sFlt‐1/PlGF immunoassay ratio substantially increased the detection rate at all gestational ages studied (assuming a false‐positive rate of both 5% and 10%) (Table 6). Table 6 Performance of standard prediction model and same model plus soluble fms‐like tyrosine kinase‐1 (sFlt‐1)/placental growth factor (PlGF) ratio to estimate risk of early‐onset pre‐eclampsia at 20, 24 and 28 weeks

We also compared the performance of a standard prediction model (maternal history, MAP and UtA‐PI) with the same model but including the sFlt‐1/PlGF immunoassay ratio to estimate early‐onset PE risk at 20, 24 and 28 weeks' gestation. The addition of the sFlt‐1/PlGF immunoassay ratio substantially increased the detection rate at all gestational ages studied (assuming a false‐positive rate of both 5% and 10%) (Table 6). Table 6 Performance of standard prediction model and same model plus soluble fms‐like tyrosine kinase‐1 (sFlt‐1)/placental growth factor (PlGF) ratio to estimate risk of early‐onset pre‐eclampsia at 20, 24 and 28 weeks Detection rate (%) Prediction model AUC (95% CI) FPR = 5% FPR = 10% 20 weeks Standard prediction model (maternal history, MAP, UtA‐PI) 0.81 (0.71–0.89) 17 48 Standard prediction model plus sFlt‐1/PlGF ratio 0.91 (0.85–0.97) 60 70 24 weeks Standard prediction model (maternal history, MAP, UtA‐PI) 0.87 (0.79–0.94) 40 60 Standard prediction model plus sFlt‐1/PlGF ratio 0.95 (0.90–0.99) 72 83 28 weeks Standard prediction model (maternal history, MAP, UtA‐PI) 0.89 (0.83–0.95) 42 58 Standard prediction model plus sFlt‐1/PlGF ratio 0.95 (0.90–1.00) 80 80 AUC, area under receiver–operating characteristics curve; FPR, false‐positive rate; MAP, mean arterial pressure; UtA‐PI, uterine artery pulsatility index.

83 28 weeks Standard prediction model (maternal history, MAP, UtA‐PI) 0.89 (0.83–0.95) 42 58 Standard prediction model plus sFlt‐1/PlGF ratio 0.95 (0.90–1.00) 80 80 AUC, area under receiver–operating characteristics curve; FPR, false‐positive rate; MAP, mean arterial pressure; UtA‐PI, uterine artery pulsatility index. sFlt‐1 and PlGF as single biomarkers: development of PE Women with early‐onset PE had lower PlGF (P < 0.001 at 20, 24 and 28 weeks) and higher sFlt‐1 (P = 0.018, P < 0.001 and P < 0.001 at 20, 24 and 28 weeks, respectively) compared with those who did not develop early‐onset PE (women who developed late‐onset PE and those who did not develop any PE combined) (Figure S4). A comparison of prediction models that included the sFlt‐1/PlGF ratio with models that used sFlt‐1 or PlGF alone was performed by evaluating their respective AUCs and Akaike information criterion (AIC), which measures goodness of fit35. The AUC for prediction models that included the sFlt‐1/PlGF ratio (0.86–0.87, 0.91–0.92 and 0.91–0.93 at 20, 24 and 28 weeks' gestation, respectively) was greater than that of models that used single biomarkers (0.81–0.83, 0.88–0.90 and 0.88–0.91 for sFlt‐1 and 0.79–0.83, 0.85–0.89 and 0.86–0.89 for PlGF at 20, 24 and 28 weeks' gestation, respectively) (Table S4). Data consistency No inconsistencies were found between site and central testing (data not shown).

A comparison of prediction models that included the sFlt‐1/PlGF ratio with models that used sFlt‐1 or PlGF alone was performed by evaluating their respective AUCs and Akaike information criterion (AIC), which measures goodness of fit35. The AUC for prediction models that included the sFlt‐1/PlGF ratio (0.86–0.87, 0.91–0.92 and 0.91–0.93 at 20, 24 and 28 weeks' gestation, respectively) was greater than that of models that used single biomarkers (0.81–0.83, 0.88–0.90 and 0.88–0.91 for sFlt‐1 and 0.79–0.83, 0.85–0.89 and 0.86–0.89 for PlGF at 20, 24 and 28 weeks' gestation, respectively) (Table S4). Data consistency No inconsistencies were found between site and central testing (data not shown). DISCUSSION Substantial evidence supports the use of the sFlt‐1/PlGF ratio in PE diagnosis and prediction20, 21, 23, 27, 36, 37, 38, 39, 40. However, differences between early‐ and late‐onset PE suggest different etiologies; thus, different ‘rules’ for the sFlt‐1/PlGF ratio could be applied. In a study of 257 women with suspected PE, the optimal sFlt‐1/PlGF ratio cut‐off to diagnose PE < 34 weeks' and ≥ 34 weeks' gestation was 23 (92.0% sensitivity, 81.1% specificity) and 45 (83.7% sensitivity, 72.6% specificity), respectively41. In PROGNOSIS, a sFlt‐1/PlGF ratio cut‐off ≤ 38 ruled out PE within 1 week in women with suspected PE and singleton pregnancy at 24 + 0 to 36 + 6 weeks' gestation32. PROGNOSIS had a higher prevalence of PE compared with our study (19% vs 11%, respectively), possibly due to the fact that PROGNOSIS enrolled women with suspicion of PE, while we enrolled women with a moderate or high risk of developing PE. The prevalence of PE in our study falls between the estimated ranges for women with moderate or high risk for PE (5.29–6.19% and 16.09–19.49%, respectively)42.

spectively), possibly due to the fact that PROGNOSIS enrolled women with suspicion of PE, while we enrolled women with a moderate or high risk of developing PE. The prevalence of PE in our study falls between the estimated ranges for women with moderate or high risk for PE (5.29–6.19% and 16.09–19.49%, respectively)42. In STEPS, the sFlt‐1/PlGF ratio was significantly different between women who did not develop PE and those who did. The combination of the sFlt‐1/PlGF ratio with other clinical measures produced a predictive model with considerably increased specificity and sensitivity compared with using UtA‐PI or sFlt‐1/PlGF ratio alone. We also evaluated how the models used to estimate early‐onset PE risk would perform when using the single biomarkers, instead of the sFlt‐1/PlGF ratio. Based on both AUC and AIC, models with the sFlt‐1/PlGF ratio demonstrated consistently the highest predictive performance. Using our early‐onset PE prediction model (sFlt‐1/PlGF ratio, MAP, being parous, previous PE), early‐onset PE could be predicted from 20 weeks onward, with an AUC of 0.86 and 60% sensitivity for a false‐positive rate of 10%. A previous model developed without serum biomarkers, which used a history of diabetes, hypertension and MAP, reported an AUC of 0.83 with 55% sensitivity for a false‐positive rate of 10% (these were not high‐risk women)43.

d from 20 weeks onward, with an AUC of 0.86 and 60% sensitivity for a false‐positive rate of 10%. A previous model developed without serum biomarkers, which used a history of diabetes, hypertension and MAP, reported an AUC of 0.83 with 55% sensitivity for a false‐positive rate of 10% (these were not high‐risk women)43. Other studies have included sFlt‐1 and PlGF in their models. An observational study of women at high risk of PE developed a prediction model for early‐onset PE using sFlt‐1 at 28 + 0 to 31 + 6 weeks' gestation, which had an AUC of 0.85 (67% sensitivity, 96% specificity)44. A model including gestational age, UtA‐PI and sFlt‐1/PlGF ratio showed an association with perinatal complications with an AUC of 0.89 (64% sensitivity, 95% specificity)45. Another model combined PlGF with maternal characteristics, obstetric history and UtA‐PI to predict early‐onset PE in the first trimester with an AUC of 0.9446. Although an abnormal UtA‐PI was associated with the development of PE in our study, it was not included in our model since it did not substantially improve PE prediction. From a practical perspective, the UtA can be difficult to locate in the first trimester and the International Society of Ultrasound in Obstetrics and Gynecology does not include UtA Doppler as part of the routine first‐trimester fetal ultrasound examination47. Other studies have also not included UtA‐PI in their models48, 49.

practical perspective, the UtA can be difficult to locate in the first trimester and the International Society of Ultrasound in Obstetrics and Gynecology does not include UtA Doppler as part of the routine first‐trimester fetal ultrasound examination47. Other studies have also not included UtA‐PI in their models48, 49. A recent study demonstrated that a prospective screening model at 19–24 weeks' gestation, involving maternal factors, UtA‐PI, MAP and PlGF, was superior to screening by maternal factors alone. The performance of the model was inversely related to the gestational age at which delivery became necessary; detection rates (false‐positive rate of 10%) for PE < 32 weeks, between 32 + 0 and 36 + 6 weeks, and ≥ 37 weeks were 99%, 85% and 46%, respectively. However, this study evaluated PlGF and sFlt‐1 separately; it did not assess the sFlt‐1/PlGF ratio50. Of note, the study defined early‐onset PE as requiring delivery before 32 weeks' gestation, rather than before 34 weeks. A related study showed that a two‐stage screening model, in which UtA‐PI and PlGF measurements were reserved for at‐risk individuals, achieved similar detection rates for preterm PE (< 37 weeks' gestation), compared with screening the whole population by maternal factors, MAP, UtA‐PI and PlGF51.

r than before 34 weeks. A related study showed that a two‐stage screening model, in which UtA‐PI and PlGF measurements were reserved for at‐risk individuals, achieved similar detection rates for preterm PE (< 37 weeks' gestation), compared with screening the whole population by maternal factors, MAP, UtA‐PI and PlGF51. Various guidelines recommend PE screening based on maternal history52, 53, 54. However, the addition of MAP, UtA‐PI and angiogenic serum markers to the assessment of maternal history has been shown to increase the PE detection rate between 12 and 36 weeks' gestation55, 56, 57, 58. In STEPS, the addition of the sFlt‐1/PlGF ratio increased the detection rate at all gestational ages studied, supporting the inclusion of the sFlt‐1/PlGF ratio in the risk estimation of early‐onset PE.

of maternal history has been shown to increase the PE detection rate between 12 and 36 weeks' gestation55, 56, 57, 58. In STEPS, the addition of the sFlt‐1/PlGF ratio increased the detection rate at all gestational ages studied, supporting the inclusion of the sFlt‐1/PlGF ratio in the risk estimation of early‐onset PE. The prospective, longitudinal design and large cohort in our study provided a robust dataset and the angiogenic marker results were hidden from the investigators to avoid bias in the diagnosis of outcomes. However, despite the large sample size, there were relatively small numbers of women in the early‐onset PE group and the results of this study, which included women at risk of developing PE, cannot be applied to a low‐risk population, i.e. in screening for PE. The data were validated using the Elecsys immunoassay sFlt‐1/PlGF ratio and the predictive value may differ when other assays are used. The developed prediction model for early‐onset PE has to be validated in an independent prospective study cohort in a comparable target population, and interventional studies are required to confirm the clinical utility of the results.

ay sFlt‐1/PlGF ratio and the predictive value may differ when other assays are used. The developed prediction model for early‐onset PE has to be validated in an independent prospective study cohort in a comparable target population, and interventional studies are required to confirm the clinical utility of the results. STEPS provides further evidence that the addition of the sFlt‐1/PlGF ratio to clinical protocols for women at risk of PE improves prediction of early‐onset PE in the second trimester. This complements the findings of PROGNOSIS, which showed that, in women with signs and/or symptoms of PE and a sFlt‐1/PlGF ratio above 38, the positive predictive value for PE within the following 4 weeks was 36.7%32. In STEPS, women who developed early‐onset PE had a median sFlt‐1/PlGF ratio at 28 weeks' gestation of 51.9, indicating that these women had an increased risk of developing PE in the following 4 weeks before 34 + 0 weeks' gestation. Better prediction of PE could facilitate targeting of monitoring and therapeutic procedures towards at‐risk women and allow better utilization of healthcare resources. DISCLOSURES The STEPS study was sponsored by Roche Diagnostics Spain who were involved in study design, interpretation of the data and writing of the manuscript. ELECSYS and cobas are trademarks of Roche. A.P. is a consultant for Roche, GE Healthcare, Ferring, Italfarmaco, EFFIK, Merck and Gynea. J.L.D. is a consultant for Roche and Italfarmaco. M.H. is employed by Roche Diagnostics and has shares in F. Hoffmann‐La Roche.

nterpretation of the data and writing of the manuscript. ELECSYS and cobas are trademarks of Roche. A.P. is a consultant for Roche, GE Healthcare, Ferring, Italfarmaco, EFFIK, Merck and Gynea. J.L.D. is a consultant for Roche and Italfarmaco. M.H. is employed by Roche Diagnostics and has shares in F. Hoffmann‐La Roche. STEPS INVESTIGATORS Azahar Romero and Francisco Cabrera, Hospital Materno Infantil de Canarias, Gran Canaria, Spain; María Vázquez, Francisco Moreno and Óscar Vaquerizo, Hospital Universitario Central de Asturias, Oviedo, Spain; Myriam Miguel, Catalina de Paco, Miriam Pertegal and Alicia Arteaga, Hospital Virgen de la Arrixaca, Murcia, Spain; M. Jesús Franco and Blanca Envid, Hospital Miguel Servet, Zaragoza, Spain; Elena Martin, José Luis Bartha and Antonio Buño, Hospital de la Paz, Madrid, Spain; Gema Pérez, Silvia Roig, Rosa Gómez and David Hervás, Hospital Universitario y Politécnico La Fe, Valencia, Spain; Ángel Aguaron de la Cruz and Nieves López, Hospital Gregorio Marañón, Madrid, Spain; Victoria Melero and Mercedes Calero, Hospital Puerta del Mar, Cadiz, Spain; Marta de Ramón and Antonio Paya, Hospital del Mar, Barcelona, Spain. Supporting information Table S1 Ethics Committee approval details Table S2 Participant recruitment by study site Table S3 Comparison of angiogenic factors at 20, 24 and 28 weeks in women with singleton vs multiple pregnancy

STEPS INVESTIGATORS Azahar Romero and Francisco Cabrera, Hospital Materno Infantil de Canarias, Gran Canaria, Spain; María Vázquez, Francisco Moreno and Óscar Vaquerizo, Hospital Universitario Central de Asturias, Oviedo, Spain; Myriam Miguel, Catalina de Paco, Miriam Pertegal and Alicia Arteaga, Hospital Virgen de la Arrixaca, Murcia, Spain; M. Jesús Franco and Blanca Envid, Hospital Miguel Servet, Zaragoza, Spain; Elena Martin, José Luis Bartha and Antonio Buño, Hospital de la Paz, Madrid, Spain; Gema Pérez, Silvia Roig, Rosa Gómez and David Hervás, Hospital Universitario y Politécnico La Fe, Valencia, Spain; Ángel Aguaron de la Cruz and Nieves López, Hospital Gregorio Marañón, Madrid, Spain; Victoria Melero and Mercedes Calero, Hospital Puerta del Mar, Cadiz, Spain; Marta de Ramón and Antonio Paya, Hospital del Mar, Barcelona, Spain. Supporting information Table S1 Ethics Committee approval details Table S2 Participant recruitment by study site Table S3 Comparison of angiogenic factors at 20, 24 and 28 weeks in women with singleton vs multiple pregnancy Table S4 Performance of different prediction models using soluble fms‐like tyrosine kinase‐1 (sFlt‐1)/placental growth factor (PlGF) ratio, sFlt‐1 or PlGF to estimate risk of early‐onset pre‐eclampsia (PE) at 20, 24 and 28 weeks Figure S1 Formulae used in prediction model for early‐onset pre‐eclampsia (PE). MAP, mean arterial pressure.

Table S4 Performance of different prediction models using soluble fms‐like tyrosine kinase‐1 (sFlt‐1)/placental growth factor (PlGF) ratio, sFlt‐1 or PlGF to estimate risk of early‐onset pre‐eclampsia (PE) at 20, 24 and 28 weeks Figure S1 Formulae used in prediction model for early‐onset pre‐eclampsia (PE). MAP, mean arterial pressure. Figure S2 Nomogram for estimation of risk of early‐onset pre‐eclampsia (PE) at 24 weeks. To calculate probability of early‐onset PE for a given patient, the value for each predictor is obtained by drawing a vertical line straight upward from that factor to the ‘points’ axis. The points are then summed and the sum located on the total points nomogram, and the probability of early‐onset PE is located by drawing a vertical line downward to the ‘risk of PE’ line. MAP, mean arterial pressure; PlGF, placental growth factor; sFlt‐1, soluble fms‐like tyrosine kinase‐1.

factor to the ‘points’ axis. The points are then summed and the sum located on the total points nomogram, and the probability of early‐onset PE is located by drawing a vertical line downward to the ‘risk of PE’ line. MAP, mean arterial pressure; PlGF, placental growth factor; sFlt‐1, soluble fms‐like tyrosine kinase‐1. Figure S3 Worked example of the use of the nomogram for estimation of risk of early‐onset pre‐eclampsia (PE) at 24 weeks. To calculate visually risk of early‐onset PE in a parous patient with previous PE at 24 weeks, with mean arterial pressure (MAP) of 100 mmHg and soluble fms‐like tyrosine kinas‐1 (sFlt‐1)/placental growth factor (PlGF) ratio of 50, points are assigned for each item by plotting a line from the item to the points line. Being parous equates to 0 points; 100 mmHg MAP corresponds to 9 points; previous PE equates to 38 points; and sFlt‐1/PlGF ratio of 50 (In = 3.91) gives 62 points. Total number of points is 109; drawing a vertical line downward from total points axis to ‘risk of PE’ line, the risk is estimated at approximately 80%. Figure S4 Placental growth factor (PlGF) (a) and soluble fms‐like tyrosine kinase‐1 (sFlt‐1) (b) at 20, 24 and 28 weeks in women who developed early‐onset pre‐eclampsia (PE) and in control participants who did not develop PE. Click here for additional data file.

Figure S3 Worked example of the use of the nomogram for estimation of risk of early‐onset pre‐eclampsia (PE) at 24 weeks. To calculate visually risk of early‐onset PE in a parous patient with previous PE at 24 weeks, with mean arterial pressure (MAP) of 100 mmHg and soluble fms‐like tyrosine kinas‐1 (sFlt‐1)/placental growth factor (PlGF) ratio of 50, points are assigned for each item by plotting a line from the item to the points line. Being parous equates to 0 points; 100 mmHg MAP corresponds to 9 points; previous PE equates to 38 points; and sFlt‐1/PlGF ratio of 50 (In = 3.91) gives 62 points. Total number of points is 109; drawing a vertical line downward from total points axis to ‘risk of PE’ line, the risk is estimated at approximately 80%. Figure S4 Placental growth factor (PlGF) (a) and soluble fms‐like tyrosine kinase‐1 (sFlt‐1) (b) at 20, 24 and 28 weeks in women who developed early‐onset pre‐eclampsia (PE) and in control participants who did not develop PE. Click here for additional data file. ACKNOWLEDGMENTS We thank all women who participated in the STEPS study and the recruitment officers, nurses, midwives and midwifery staff who supported the study. We thank the STEPS investigators and Mª José Ramirez and Nuria Piella (Roche Diagnostics, Spain). Support for third‐party writing assistance for this manuscript was provided by Emma McConnell, PhD (Gardiner‐Caldwell Communications), and was funded by Roche Diagnostics.

The accuracy of targeted cell‐free DNA (cfDNA) testing with DANSR™ and FORTE™ for trisomies 21, 18 and 13 has been well demonstrated and is consistent across next generation sequencing and microarray quantitation methods1. Targeted cfDNA analysis for fetal sex chromosome aneuploidy (SCA) has also been validated and shown to have high specificity in prospective studies2, 3. This study expands upon the available published data by investigating the performance of targeted cfDNA analysis of the X and Y chromosomes using microarray quantitation for assessment of SCA probability in singleton pregnancy and fetal sex in twin and singleton pregnancies. Samples of banked maternal plasma from 791 singleton and 51 twin pregnancies were obtained as part of ongoing multicenter clinical studies (NCT02201862 and NCT01451671) and from a sample bank at King's College London, UK. Single cell‐free Roche, Streck BCT‐DNA or EDTA collection tubes were available for each sample. Collection and processing differed from commercial protocols only in that all samples were frozen prior to analysis and available specimen volumes were lower than standardly used. Patient consent and fetal karyotype information was obtained for all samples. The cohort included 15 singleton pregnancies with sex chromosome aneuploidy. Targeted cfDNA analysis with microarray quantitation was performed, as previously described, using a blinded protocol4.

imen volumes were lower than standardly used. Patient consent and fetal karyotype information was obtained for all samples. The cohort included 15 singleton pregnancies with sex chromosome aneuploidy. Targeted cfDNA analysis with microarray quantitation was performed, as previously described, using a blinded protocol4. Y‐chromosome specific DANSR assays were used to evaluate fetal sex in twin and singleton pregnancies. Results were reported as male or female, depending on concluded presence or absence of Y‐chromosome fragments. In twin pregnancies, a male result indicates the presence of at least one male fetus. Fetal SCA analysis was performed on samples from singleton pregnancies using X‐ and Y‐specific DANSR assays followed by FORTE analysis adapted for this purpose2, 4, 5. A probability cut‐off of 1 in 100 for non‐disomic genotypes was used for calculation of sensitivity and specificity. Gestational age and fetal fraction averaged 16.7 weeks and 13.4%, respectively. Thirty‐nine singleton and 12 twin pregnancy samples had insufficient fetal fraction or failed to pass quality control thresholds resulting in 752 and 39 samples undergoing testing for fetal sex in singleton and twin pregnancies, respectively. Fetal sex results were yielded in 748/752 singleton and 39/39 twin samples. Of these, predicted fetal sex was consistent with karyotypic sex in 786/787 cases (99.9% concordance) (Table 1). All twin fetal sex cfDNA results accurately reflected either the presence of two female fetuses (n = 18) or at least one male fetus (n = 21).

results were yielded in 748/752 singleton and 39/39 twin samples. Of these, predicted fetal sex was consistent with karyotypic sex in 786/787 cases (99.9% concordance) (Table 1). All twin fetal sex cfDNA results accurately reflected either the presence of two female fetuses (n = 18) or at least one male fetus (n = 21). Table 1 Performance of cell‐free DNA (cfDNA) testing for fetal sex in singleton and twin pregnancy and for sex chromosome aneuploidy in singleton pregnancy in current study and previous publications Study Fetal sex accuracy* Sex chromosome aneuploidy Disomy accuracy* 45,X† 47,XXX† 47,XXY† Singleton Twin DR FPR DR FPR DR FPR Current 747/748 (99.9 (99.3–100)) 39/39 (100 (91.0–100)) 13/13 (100 (77.2–100)) 1/742 (0.1 (0–0.8)) 1/1 1/742 (0.1 (0–0.8)) 1/1 0/742 (0 (0–0.5)) 725/727 (99.7 (99.0–99.9)) Hooks2 414/414 (100 (99.1–100)) — 26/27 (96.3 (81.7–99.3)) 2/380 (0.5 (0.2–1.9)) 1/1 2/380 (0.5 (0.2–1.9)) 6/6 0/380 (0 (0–1)) 378/380 (99.5 (98.1–99.9)) Nicolaides3 109/110 (99.1 (95–99.8)) — 43/47 (91.5 (80–96.6)) 0/172 (0 (0–2.2)) 5/5 1/172 (0.6 (0.1–3.2)) 1/1 0/172 (0 (0–2.2)) 115/116 (99.1 (95.3–99.9)) Only first author of each study is given. Data are presented as n/N (% (95% CI)). * Accuracy defined as concordant cfDNA and karyotype test results. † Sensitivities for individual sex chromosome aneuploidies cannot be concluded due to small number of affected pregnancies. DR, detection rate; FPR, false‐positive rate.

Singleton Twin DR FPR DR FPR DR FPR Current 747/748 (99.9 (99.3–100)) 39/39 (100 (91.0–100)) 13/13 (100 (77.2–100)) 1/742 (0.1 (0–0.8)) 1/1 1/742 (0.1 (0–0.8)) 1/1 0/742 (0 (0–0.5)) 725/727 (99.7 (99.0–99.9)) Hooks2 414/414 (100 (99.1–100)) — 26/27 (96.3 (81.7–99.3)) 2/380 (0.5 (0.2–1.9)) 1/1 2/380 (0.5 (0.2–1.9)) 6/6 0/380 (0 (0–1)) 378/380 (99.5 (98.1–99.9)) Nicolaides3 109/110 (99.1 (95–99.8)) — 43/47 (91.5 (80–96.6)) 0/172 (0 (0–2.2)) 5/5 1/172 (0.6 (0.1–3.2)) 1/1 0/172 (0 (0–2.2)) 115/116 (99.1 (95.3–99.9)) Only first author of each study is given. Data are presented as n/N (% (95% CI)). * Accuracy defined as concordant cfDNA and karyotype test results. † Sensitivities for individual sex chromosome aneuploidies cannot be concluded due to small number of affected pregnancies. DR, detection rate; FPR, false‐positive rate. For SCA assessment, 742 samples were eligible. All 15 cases of SCAs were correctly identified (100% sensitivity; 95% CI, 79.6–100%) (Table 1). Out of 727 disomic (XX or XY) pregnancies, 725 were correctly classified as low‐risk for SCA (99.7% specificity; 95% CI, 99.0–99.9%) (Table 1).

DR, detection rate; FPR, false‐positive rate. For SCA assessment, 742 samples were eligible. All 15 cases of SCAs were correctly identified (100% sensitivity; 95% CI, 79.6–100%) (Table 1). Out of 727 disomic (XX or XY) pregnancies, 725 were correctly classified as low‐risk for SCA (99.7% specificity; 95% CI, 99.0–99.9%) (Table 1). In summary, targeted cfDNA analysis performed with high accuracy for fetal sex assessment in twins and singletons, and correctly identified all SCAs with high specificity. A limitation of using these banked samples is that the positive predictive value observed in this enriched cohort would not be translatable to a routine prenatal screening population. In addition, the number of samples passing quality thresholds may be lower than standard due to irregular sample volumes. However, this study provides a valuable supplement to the currently available data supporting the use of targeted cfDNA analysis for fetal sex and SCA assessment and substantiates previous conclusions that the performance of this methodology is robust across quantitation platforms. Disclosures With the exception of K.H.N., all authors are employees of Roche Sequencing Solutions. The study was funded by Roche Sequencing Solutions.

INTRODUCTION In fetal growth restriction (FGR), Doppler ultrasound examination can be used to distinguish between fetuses that are at risk of adverse perinatal outcome and those that are constitutionally small. Current FGR guidelines recommend umbilical artery (UA) Doppler as an important surveillance tool1, 2, 3, since its clinical effectiveness in high‐risk pregnancies has been reported in a Cochrane review of 18 randomized controlled trials4. Fetal middle cerebral artery (MCA) Doppler has been proposed as an additional test to UA Doppler. Decreased impedance in the MCA is considered the ‘brain‐sparing’ effect; a manifestation of redistribution of the fetal circulation and possibly a sign of compromise. A systematic review of MCA Doppler reported limited summary likelihood ratios for predicting adverse perinatal outcome5. The cerebroplacental ratio (CPR) is calculated as the ratio of MCA to UA Doppler, and has been hypothesized to be more accurate than its individual components6. An association with adverse perinatal outcome has been the focus of several literature reviews7, 8, 9, 10. CPR and MCA Doppler in FGR are gradually becoming integrated into clinical practice and international guidelines1, 3, 11, but reported estimates of their accuracy vary considerably. We conducted a systematic review and meta‐analysis of published studies on the prognostic accuracy of CPR and MCA Doppler compared with UA Doppler in the prediction of adverse perinatal outcome, in order to identify whether CPR and MCA Doppler evaluation are of added value to UA Doppler.

in digital format (Echopac®, GE Medical Systems). Echocardiographic data were analyzed retrospectively for ventricular and valvular dimensions and ratios, as well as for RV filling time (duration of tricuspid valve (TV) inflow/cardiac cycle length), TV velocity time integral (TV‐VTI) × heart rate (HR), and TR velocity. All interventions were performed under ultrasound guidance as described previously8, 14 and under general anesthesia of the mother without separate fetal analgesia. The fetal RV was punctured with a 16‐, 18‐ or 19‐gauge needle (Cook® Medical Systems, Limerick, Ireland). A 3‐, 5‐ or 4‐mm coronary balloon catheter (Maverick®, Boston Scientific, Vienna, Austria) was used for valve dilatation. The balloon‐to‐valve ratio was aimed at between 1 and 1.5; however, balloon size was limited by the inner diameter of the needle. Eight patients had two and two patients had three procedures due to technical failure of the first attempts. A procedure was considered successful if the PV was perforated and/or passed and dilated with a balloon catheter. A partially successful procedure was defined as one in which the PV was perforated and passed, but the valve was not dilated with the full diameter of the balloon.

Fetal middle cerebral artery (MCA) Doppler has been proposed as an additional test to UA Doppler. Decreased impedance in the MCA is considered the ‘brain‐sparing’ effect; a manifestation of redistribution of the fetal circulation and possibly a sign of compromise. A systematic review of MCA Doppler reported limited summary likelihood ratios for predicting adverse perinatal outcome5. The cerebroplacental ratio (CPR) is calculated as the ratio of MCA to UA Doppler, and has been hypothesized to be more accurate than its individual components6. An association with adverse perinatal outcome has been the focus of several literature reviews7, 8, 9, 10. CPR and MCA Doppler in FGR are gradually becoming integrated into clinical practice and international guidelines1, 3, 11, but reported estimates of their accuracy vary considerably. We conducted a systematic review and meta‐analysis of published studies on the prognostic accuracy of CPR and MCA Doppler compared with UA Doppler in the prediction of adverse perinatal outcome, in order to identify whether CPR and MCA Doppler evaluation are of added value to UA Doppler. METHODS Search strategy and selection criteria This systematic review is described in accordance with the Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA) statement (www.prisma‐statement.org). The protocol for this systematic review was registered in the PROSPERO database (CRD42016039915).

We conducted a systematic review and meta‐analysis of published studies on the prognostic accuracy of CPR and MCA Doppler compared with UA Doppler in the prediction of adverse perinatal outcome, in order to identify whether CPR and MCA Doppler evaluation are of added value to UA Doppler. METHODS Search strategy and selection criteria This systematic review is described in accordance with the Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA) statement (www.prisma‐statement.org). The protocol for this systematic review was registered in the PROSPERO database (CRD42016039915). A comprehensive search of PubMed, EMBASE, the Cochrane Library (via Wiley) and ClinicalTrials.gov was performed from inception to June 14th 2016, in collaboration with a medical librarian. Search terms included controlled terms (MeSH in PubMed, Emtree in EMBASE) as well as free‐text terms (Cochrane Library only). The following terms were used as index terms or free‐text words (including synonyms and closely related words): (‘pregnancy’ or ‘fetus’) and (‘middle cerebral artery’ or ‘CPR’ or ‘brain sparing’). The full search strategies for all the databases can be found in Appendix S1. Duplicate articles were excluded and reference lists of included articles and previous reviews scanned for further eligible articles. Articles were not filtered based on language.

’) and (‘middle cerebral artery’ or ‘CPR’ or ‘brain sparing’). The full search strategies for all the databases can be found in Appendix S1. Duplicate articles were excluded and reference lists of included articles and previous reviews scanned for further eligible articles. Articles were not filtered based on language. We searched for studies that assessed the prognostic accuracy of UA Doppler compared with CPR and/or MCA Doppler in women with a singleton pregnancy of any risk profile, without known chromosomal or structural abnormalities. Studies were eligible if they reported on the association between Doppler indices and at least one of the following adverse perinatal outcomes: perinatal death, 5‐min Apgar score < 7, acidosis on cord blood gas analysis, abnormal cardiotocogram, emergency delivery for fetal distress, meconium‐stained fluid, need for assisted respiration after birth, need for resuscitation, admission to the neonatal intensive care unit (NICU) or neonatal unit, serious neonatal morbidity (e.g. necrotizing enterocolitis, sepsis, respiratory distress syndrome), neurological outcomes, ultrasound‐detected neonatal intracranial abnormalities, a composite of adverse outcomes (as defined in the included studies) or birth weight (BW) < 10th percentile. In cases of multiple publications from one study group, we included the data reporting on the largest study group. Studies reporting on UA Doppler but not on MCA Doppler or CPR were excluded, as were review articles and case reports.

outcomes (as defined in the included studies) or birth weight (BW) < 10th percentile. In cases of multiple publications from one study group, we included the data reporting on the largest study group. Studies reporting on UA Doppler but not on MCA Doppler or CPR were excluded, as were review articles and case reports. Data extraction and quality assessment Two reviewers (C.A.V. and M.A.B.) independently assessed titles and abstracts of all search results. Full reports of studies considered potentially eligible by at least one of the reviewers were obtained. The same two reviewers (C.A.V. and M.A.B.) assessed the full reports for inclusion and extracted data. In case of doubt, other reviewers were consulted (C.J.B., B.W.J.M. or C.J.M.G.). From each included study report, we identified the first author, country, journal, year of publication, recruitment setting, sample size and characteristics of included patients (age, gestational age at test and at delivery, fetal growth and gestational hypertensive disorders). We extracted data on the index test (including Doppler index, and threshold and reference values), targeted outcomes, accuracy estimates and data for 2 × 2 tables. Whenever UA Doppler results were presented, we also extracted data for 2 × 2 tables. Risk of bias and concerns about applicability were assessed by two authors (C.A.V. and C.J.B.) with the QUADAS‐2 (Quality Assessment of Diagnostic Accuracy Studies‐2) tool12.

o technical failure of the first attempts. A procedure was considered successful if the PV was perforated and/or passed and dilated with a balloon catheter. A partially successful procedure was defined as one in which the PV was perforated and passed, but the valve was not dilated with the full diameter of the balloon. Patients' charts were reviewed for postnatal procedures and outcome. Biventricular outcome was defined as a circulation in which the RV was the only source of pulmonary blood flow in the absence of any signs of right‐heart failure. Undetermined circulation was one in which a modified Blalock–Taussig (BT) shunt or ductal stent was placed postnatally and the final circulation was still unclear. Eight patients were delivered and treated in our center, whereas all other patients were delivered and managed in their respective home countries. For the longitudinal assessment of RV growth, only measurements from fetuses that were delivered and followed up at our center were used.

eted outcomes, accuracy estimates and data for 2 × 2 tables. Whenever UA Doppler results were presented, we also extracted data for 2 × 2 tables. Risk of bias and concerns about applicability were assessed by two authors (C.A.V. and C.J.B.) with the QUADAS‐2 (Quality Assessment of Diagnostic Accuracy Studies‐2) tool12. Statistical analysis Meta‐analyses were performed for the outcomes perinatal death, 5‐min Apgar score < 7, emergency delivery for fetal distress, NICU admission and composite adverse perinatal outcome (as defined in the included studies). These were selected by consensus, as they are the most homogeneously classified and least subject to ascertainment bias. Owing to variation in thresholds across studies, the hierarchal summary receiver–operating characteristics (HSROC) model of Rutter and Gatsonis13, 14 was used to obtain summary ROC curves of the CPR and MCA Doppler for studies that had evaluated the same target outcome. The HSROC model allows for variation of accuracy and thresholds between studies. One threshold per study was selected to be included in this analysis. If a study reported on more than one threshold, the most commonly used Doppler index (i.e. pulsatility index (PI)) and threshold (i.e. for CPR < 1.08 and for MCA Doppler < 2 SD below the mean) were selected. Whenever three or more studies were available, estimates of mean sensitivity and specificity and respective variances at a specific threshold were additionally generated using the bivariate logit‐normal model15.

ity index (PI)) and threshold (i.e. for CPR < 1.08 and for MCA Doppler < 2 SD below the mean) were selected. Whenever three or more studies were available, estimates of mean sensitivity and specificity and respective variances at a specific threshold were additionally generated using the bivariate logit‐normal model15. The prognostic test accuracy of CPR vs MCA Doppler was compared indirectly, using a HSROC meta‐regression model by including test type as a covariate. This meta‐regression approach was also used to investigate heterogeneity, by adding potential sources of heterogeneity as covariates to the HSROC model. A priori considered covariates were: fetal growth (small‐for‐gestational age (SGA) and appropriate‐for‐gestational age (AGA)), duration of pregnancy (preterm and term) and timing of measurement (within 1 week of delivery and more than 1 week from delivery). Preterm delivery was defined as delivery before 37 weeks' gestation. SGA was defined as estimated fetal weight (EFW) or BW < 10th percentile.

GA) and appropriate‐for‐gestational age (AGA)), duration of pregnancy (preterm and term) and timing of measurement (within 1 week of delivery and more than 1 week from delivery). Preterm delivery was defined as delivery before 37 weeks' gestation. SGA was defined as estimated fetal weight (EFW) or BW < 10th percentile. The prognostic test accuracies of CPR and MCA Doppler were then compared directly with that of UA Doppler using a HSROC meta‐regression model. For this purpose, analyses were restricted to studies that had compared two tests in the same study group, which ensures that differences in accuracy are not caused by heterogeneity in performance across study populations. Review Manager (RevMan) version 5.3 (Cochrane Collaboration, Copenhagen, 2014) was used for data extraction and generating forest plots and summary ROC curves. SAS® Studio (SAS Institute Inc., Cary, NC, USA) was used for HSROC model analysis and Stata version 14 (StataCorp LP, College Station, TX, USA) for the bivariate models. RESULTS Search results A total of 4689 articles were identified through the electronic search, and four additional articles through cross‐referencing (Figure 1). After removing duplicates, the remaining 2972 titles and abstracts were screened, resulting in 578 potentially eligible articles. Of these, 450 articles could not be included, for reasons listed in Figure 1. This resulted in 128 studies (involving 47 748 women) being included (Appendix S2), of which 52 studies evaluated CPR only, 48 studies MCA Doppler only and 28 studies both CPR and MCA Doppler.

, resulting in 578 potentially eligible articles. Of these, 450 articles could not be included, for reasons listed in Figure 1. This resulted in 128 studies (involving 47 748 women) being included (Appendix S2), of which 52 studies evaluated CPR only, 48 studies MCA Doppler only and 28 studies both CPR and MCA Doppler. Figure 1 Flowchart of studies included in systematic review and meta‐analysis on prognostic accuracy of cerebroplacental ratio (CPR) and/or middle cerebral artery (MCA) Doppler in prediction of adverse perinatal outcome in singleton pregnancies compared with umbilical artery (UA) Doppler. UOG-18809-FIG-0001-bCharacteristics of included studies Detailed characteristics of the included studies are provided in Table S1. Data had been prospectively collected in 73 studies (57%), retrospectively in 25 studies (20%), and by unclear methods in 30 studies (23%). The number of patients included in the different studies ranged from 19 to 9198. Mean or median maternal age ranged from 23.0 to 37.7 years. Definitions of FGR varied, ranging from EFW < 3rd percentile with abnormal UA Doppler to BW < 10th percentile. For this review, we combined these definitions into SGA. Most studies investigated SGA fetuses (65 studies, 51%), while 15 studies (12%) investigated only AGA fetuses. In most studies, gestational age at examination was unknown or there was a range of gestational ages (78 studies, 61%), preterm pregnancies were investigated in 28 studies (22%) and term pregnancies were investigated in 22 (17%) studies.

A fetuses (65 studies, 51%), while 15 studies (12%) investigated only AGA fetuses. In most studies, gestational age at examination was unknown or there was a range of gestational ages (78 studies, 61%), preterm pregnancies were investigated in 28 studies (22%) and term pregnancies were investigated in 22 (17%) studies. Quality of included studies Results of the QUADAS‐2 assessment are provided in Appendix S3. Risk of bias or suboptimal reporting was detected in 120/128 studies (94%). The interval between index test and outcome was often larger than 1 week (53 studies (41%)) or unclear (37 studies (29%)). Methods for patient selection were also often unclear (36 studies (28%)). In 64 studies (50%) it was unclear whether the obstetrician was blinded to the test results, and in 30 studies (23%) they were not blinded; nine of these described if and how an abnormal test result influenced obstetric decision‐making.

s (29%)). Methods for patient selection were also often unclear (36 studies (28%)). In 64 studies (50%) it was unclear whether the obstetrician was blinded to the test results, and in 30 studies (23%) they were not blinded; nine of these described if and how an abnormal test result influenced obstetric decision‐making. Meta‐analysis and indirect comparisons of CPR and MCA Doppler For the meta‐analysis, we could include 106/128 studies assessing one of the five primary outcomes defined in the Methods section. In Figure 2, summary curves of all included studies estimated with the HSROC model are shown for CPR and MCA Doppler for the outcomes perinatal death, 5‐min Apgar score < 7, emergency delivery for fetal distress, NICU admission and composite adverse perinatal outcomes (as defined in the included studies). Forest plots of all other outcomes can be found in Figure S1. Clearly observable is the large variation in sensitivity and specificity. For perinatal death, the CPR was evaluated in 25 studies (298/8003 patients) and MCA Doppler in 28 studies (326/12 230 patients). For low Apgar score, CPR was evaluated in 28 studies (751/6904 patients) and MCA Doppler in 23 studies (329/12 889 patients). For emergency delivery for fetal distress, CPR was evaluated in 28 studies (3007/15 834 patients), and MCA Doppler in 26 studies (1745/14 171 patients). For NICU admission, CPR was evaluated in 28 studies (1632/5417 patients), and MCA Doppler in 20 studies (695/2224 patients). For composite adverse outcome, CPR was evaluated in 31 studies (1574/5271 patients), and MCA Doppler in 28 studies (1445/5477 patients). In these indirect comparisons, CPR performed better than did MCA Doppler for all outcomes (composite adverse outcome, P < 0.001; perinatal death, P = 0.004; emergency delivery for fetal distress, P = 0.004; and low Apgar score, P = 0.033), except for admission to the NICU (P = 0.761).

n 28 studies (1445/5477 patients). In these indirect comparisons, CPR performed better than did MCA Doppler for all outcomes (composite adverse outcome, P < 0.001; perinatal death, P = 0.004; emergency delivery for fetal distress, P = 0.004; and low Apgar score, P = 0.033), except for admission to the NICU (P = 0.761). Figure 2 Hierarchal summary receiver–operating characteristics curves and P‐values for indirect comparisons of prognostic accuracy of cerebroplacental ratio (, black line) and middle cerebral artery Doppler (, green line) for outcomes perinatal death, 5‐min Apgar score < 7, emergency delivery (ED) for fetal distress, admission to neonatal intensive care unit (NICU) and composite adverse perinatal outcome (as defined in included studies).

accuracy of cerebroplacental ratio (, black line) and middle cerebral artery Doppler (, green line) for outcomes perinatal death, 5‐min Apgar score < 7, emergency delivery (ED) for fetal distress, admission to neonatal intensive care unit (NICU) and composite adverse perinatal outcome (as defined in included studies). UOG-18809-FIG-0002-cThresholds The asymmetric shape of the curves in Figure 2 suggests that accuracy varies with threshold. Many different thresholds were used in the studies, of which most were predefined (120 studies, 94%). Some studies used absolute thresholds, while others used percentiles on a reference curve (Appendix S4). Table 1 shows mean sensitivities and specificities, with their confidence and prediction intervals, of CPR and MCA Doppler at thresholds that were assessed in at least three studies for the outcomes perinatal death, 5‐min Apgar score < 7, emergency delivery for fetal distress, NICU admission and composite adverse perinatal outcomes (as defined in the included studies). For this purpose, absolute thresholds were grouped from 1.0 to 1.1. The thresholds that could be taken together were CPR‐PI 1.0–1.1, CPR resistance index 1.0–1.1, MCA‐PI < 2 SD on the reference curve of Mari and Deter16, and MCA‐PI < 5th percentile on the reference curve of Arduini and Rizzo17. Mean sensitivity of the CPR ranged from 0.55 to 0.93, and mean specificity ranged from 0.63 to 0.91. Mean sensitivity of MCA Doppler ranged from 0.34 to 0.67, and mean specificity ranged from 0.65 to 0.96. Prediction intervals were large, which indicates high between‐study variability in sensitivity and specificity, and in thresholds.

of the CPR ranged from 0.55 to 0.93, and mean specificity ranged from 0.63 to 0.91. Mean sensitivity of MCA Doppler ranged from 0.34 to 0.67, and mean specificity ranged from 0.65 to 0.96. Prediction intervals were large, which indicates high between‐study variability in sensitivity and specificity, and in thresholds. Table 1 Mean accuracy estimates of cerebroplacental ratio (CPR) and middle cerebral artery (MCA) Doppler for different adverse outcomes, based on fixed thresholds investigated in at least three studies

of the CPR ranged from 0.55 to 0.93, and mean specificity ranged from 0.63 to 0.91. Mean sensitivity of MCA Doppler ranged from 0.34 to 0.67, and mean specificity ranged from 0.65 to 0.96. Prediction intervals were large, which indicates high between‐study variability in sensitivity and specificity, and in thresholds. Table 1 Mean accuracy estimates of cerebroplacental ratio (CPR) and middle cerebral artery (MCA) Doppler for different adverse outcomes, based on fixed thresholds investigated in at least three studies Index test; threshold Outcome Studies (n) Patients (n) Sensitivity Specificity Estimated mean 95% CI 95% predict int Estimated mean 95% CI 95% predict int CPR‐PI; 1.0–1.1 Perinatal death 10 3571    0.93 0.71–0.99 0.15–0.99 0.74 0.62–0.84 0.32–0.95 5‐min Apgar score < 7 9 1370    0.61 0.53–0.68 0.47–0.73 0.76 0.69–0.82 0.55–0.89 ED for fetal distress 8 2559    0.58 0.39–0.74 0.15–0.92 0.89 0.79–0.94 0.52–0.98 NICU admission 12 2603    0.55 0.44–0.65 0.24–0.83 0.85 0.79–0.90 0.60–0.95 Composite adverse outcome 13 2591    0.59 0.44–0.73 0.15–0.92 0.91 0.82–0.96 0.39–0.99 CPR‐RI; 1.0–1.1 Perinatal death 5 450    0.84 0.63–0.94 0.45–0.97 0.67 0.45–0.84 0.19–0.95 5‐min Apgar score < 7 8 1782    0.75 0.42–0.93 0.06–0.99 0.84 0.63–0.94 0.16–0.99 NICU admission 4 299    0.66 0.46–0.82 0.30–0.76 0.63 0.49–0.76 0.36–0.84 Composite adverse outcome 4 409    0.80 0.69–0.89 0.59–0.93 0.85 0.66–0.94 0.40–0.98 MCA‐PI; < 2 SD16 Perinatal death 3 92    0.48 0.28–0.68 0.27–0.69 0.72 0.53–0.85 0.40–0.91 5‐min Apgar score < 7 3 125    0.56 0.35–0.75 0.32–0.77 0.65 0.45–0.80 0.31–0.88 ED for fetal distress 3 167    0.52 0.36–0.68 0.28–0.75 0.84 0.51–0.97 0.25–0.99 NICU admission 5 385    0.45 0.33–0.57 0.23–0.69 0.93 0.84–0.97 0.70–0.99 Composite adverse outcome 3 211    0.34 0.18–0.55 0.10–0.69 0.96 0.77–0.99 0.45–1.00 MCA‐PI; < 5th percentile17 ED for fetal distress 3 416    0.35 0.22–0.49 0.16–0.59 0.83 0.78–0.87 0.77–0.87 NICU admission 3 270    0.64 0.10–0.99 0.02–0.99 0.75 0.67–0.81 0.51–0.89 Composite adverse outcome 3 511    0.67 0.58–0.74 0.55–0.75 0.78 0.59–0.89 0.39–0.95 ED, emergency delivery; NICU, neonatal intensive care unit; PI, pulsatility index; predict int, prediction interval; RI, resistance index.

83 0.78–0.87 0.77–0.87 NICU admission 3 270    0.64 0.10–0.99 0.02–0.99 0.75 0.67–0.81 0.51–0.89 Composite adverse outcome 3 511    0.67 0.58–0.74 0.55–0.75 0.78 0.59–0.89 0.39–0.95 ED, emergency delivery; NICU, neonatal intensive care unit; PI, pulsatility index; predict int, prediction interval; RI, resistance index. Heterogeneity between subgroups We found evidence of heterogeneity (Table S2) in the overall test performance of the CPR related to fetal growth (SGA vs AGA), with only one out of five outcomes showing a statistically significant different accuracy (in favor of AGA). The forest plots (Figure S1) show that, in general, sensitivity is lower and specificity is similar in AGA studies compared with SGA studies. We also found evidence of heterogeneity (Table S2) in the overall test performance of the CPR and MCA Doppler related to time of pregnancy (preterm vs at term), with only two out of five outcomes showing statistically significantly different accuracy (in favor of preterm pregnancies). We found no statistical evidence of heterogeneity related to timing of the Doppler measurement (within or beyond 1 week of delivery). Other potential sources of heterogeneity, such as late FGR, could not be evaluated statistically because of the limited number of studies that reported on these covariates and owing to heterogeneity in outcome reporting. In late SGA (≥ 32 weeks' gestation), nine studies investigated the prognostic accuracy of the CPR and eight investigated the prognostic accuracy of MCA Doppler for different outcomes. These studies are summarized in Appendix S5. For the CPR, reported sensitivity ranged from 0.08 to 0.71, while specificity ranged from 0.47 to 0.98. For MCA Doppler, reported sensitivity ranged from 0.13 to 1.00, while specificity ranged from 0.67 to 0.97.

the prognostic accuracy of MCA Doppler for different outcomes. These studies are summarized in Appendix S5. For the CPR, reported sensitivity ranged from 0.08 to 0.71, while specificity ranged from 0.47 to 0.98. For MCA Doppler, reported sensitivity ranged from 0.13 to 1.00, while specificity ranged from 0.67 to 0.97. Direct comparisons of CPR, MCA Doppler and UA Doppler Figure 3 shows direct comparisons of the prognostic accuracy of CPR, MCA Doppler and UA Doppler for the outcomes perinatal death, 5‐min Apgar score < 7, emergency delivery for fetal distress, NICU admission and composite adverse perinatal outcomes (as defined in the included studies). Overall, the CPR and UA Doppler were compared in 30 studies (5046 patients), MCA Doppler and UA Doppler were compared in 38 studies (18 999 patients), and the CPR and MCA Doppler were compared in 23 studies (4262 patients). The connecting lines between the pair of tests from each study appear to show that sensitivity of CPR outperforms that of UA Doppler and sensitivity of MCA Doppler performs less well than do those of UA Doppler and CPR. A meta‐regression analysis was performed to compare statistically prognostic accuracy of the three tests in the studies that had performed a direct comparison. The CPR was significantly better than UA Doppler in predicting composite adverse outcome (P < 0.001) and emergency delivery for fetal distress (P = 0.003), but comparable with UA Doppler in predicting perinatal death (P = 0.686), low Apgar score (P = 0.595) and NICU admission (P = 0.107). MCA Doppler was significantly worse than UA Doppler in predicting low Apgar score (P = 0.017) and emergency delivery for fetal distress (P = 0.034) and significantly worse than CPR in predicting composite adverse outcome (P < 0.001) and emergency delivery for fetal distress (P = 0.013).

and NICU admission (P = 0.107). MCA Doppler was significantly worse than UA Doppler in predicting low Apgar score (P = 0.017) and emergency delivery for fetal distress (P = 0.034) and significantly worse than CPR in predicting composite adverse outcome (P < 0.001) and emergency delivery for fetal distress (P = 0.013). Figure 3 Hierarchal summary receiver–operating characteristics curves and P‐values for direct comparisons of prognostic accuracy of cerebroplacental ratio (), middle cerebral artery Doppler () and umbilical artery Doppler () for outcomes perinatal death, 5‐min Apgar score < 7, emergency delivery (ED) for fetal distress, admission to neonatal intensive care unit (NICU) and composite adverse perinatal outcome (as defined in included studies). Analyses restricted to studies that compared both tests in the same patients. Lines connect pairs of points representing the two tests from each study.

, emergency delivery (ED) for fetal distress, admission to neonatal intensive care unit (NICU) and composite adverse perinatal outcome (as defined in included studies). Analyses restricted to studies that compared both tests in the same patients. Lines connect pairs of points representing the two tests from each study. UOG-18809-FIG-0003-cDISCUSSION Summary of main results We reviewed systematically studies on the prognostic accuracy of CPR and/or MCA Doppler compared with UA Doppler in predicting adverse perinatal outcome, to ascertain whether CPR and MCA Doppler evaluation are of added value to UA Doppler. Formal quality assessment of the included studies revealed few studies of high quality. We observed a large variation in thresholds used and in reported sensitivities and specificities. In the direct test comparisons, prognostic accuracy of the CPR significantly outperformed that of UA Doppler for emergency delivery for fetal distress and composite adverse outcome, while for the other outcomes CPR was similar to UA Doppler. For all outcomes, sensitivity of CPR appeared to be better than that of UA Doppler. Prognostic accuracy of MCA Doppler was significantly worse than those of UA Doppler and CPR for most outcomes.

rgency delivery for fetal distress and composite adverse outcome, while for the other outcomes CPR was similar to UA Doppler. For all outcomes, sensitivity of CPR appeared to be better than that of UA Doppler. Prognostic accuracy of MCA Doppler was significantly worse than those of UA Doppler and CPR for most outcomes. Strengths and limitations of the study The main strength of this review is that its conclusions are based on direct within‐study comparisons in a large dataset, which limits confounding. However, several limitations deserve consideration. Subgroup analyses were limited owing to the small number of studies that explicitly described inclusion criteria and heterogeneity in outcome reporting. Almost all included studies were at risk of bias and had suboptimal reporting. Caregivers in almost all studies were aware of the Doppler results, which may have influenced obstetric decision‐making. Another concern is that publication bias may have led to overestimation of accuracy. Although no appropriate method exists to quantify publication bias in test accuracy reviews18, studies with higher sensitivities and specificities presumably have better chances of being published than those with lower sensitivities and specificities. Furthermore, the optimal test, threshold and outcome may have been reported selectively.

ate method exists to quantify publication bias in test accuracy reviews18, studies with higher sensitivities and specificities presumably have better chances of being published than those with lower sensitivities and specificities. Furthermore, the optimal test, threshold and outcome may have been reported selectively. The large differences in test accuracy between studies indicate substantial heterogeneity, for which several sources should be considered. Study populations varied, ranging from early severe FGR to uncomplicated term pregnancies. The frequency of adverse perinatal outcomes also fluctuated. Another source of heterogeneity is the large variety of thresholds used for test positivity across studies. Several methods were used across studies to classify adverse perinatal outcome and we performed a meta‐analysis only for those that could be classified homogeneously, aiming to minimize this source of heterogeneity.

a comparison between plasma and serum measurements for the DELFIA Xpress PlGF 1‐2‐3 test and the Elecsys immunoassay sFlt‐1/PlGF assay was performed. To allow direct comparison of NPV and PPV across tests, all NPVs and PPVs were calculated using an assumed prevalence of 15% for preterm pre‐eclampsia in this population. A power calculation was performed with the aim of showing equivalence, defined as a difference of less than 0.05 in the AUC between assays. Based on a prevalence of pre‐eclampsia of 15%, we planned that the study should have a power of 94.2% to detect or reject a difference between the predictive power of the tests, as measured by a difference in AUC of 0.05 or more. Demographic data are presented as median (interquartile range) or n (%). Normality of distribution was explored using a Q‐Q plot, and logarithmic transformations were used where appropriate. t‐tests or the Mann–Whitney U‐test, and Fisher's exact test were used to test differences between groups. Statistical analysis was performed using statistical package Stata Version 13 (StataCorp., College Station, TX, USA) and SPSS Version 21 (IBM Corp., Armonk, NY, USA).

ther source of heterogeneity is the large variety of thresholds used for test positivity across studies. Several methods were used across studies to classify adverse perinatal outcome and we performed a meta‐analysis only for those that could be classified homogeneously, aiming to minimize this source of heterogeneity. Implications for practice In current practice, the management of FGR is aimed at monitoring fetal condition in order to time optimally induction of delivery. In early FGR (< 32 weeks' gestation), clinical management is guided primarily by Doppler measurements of the UA and ductus venosus1. In late FGR (≥ 32 weeks), it has been suggested that CPR and MCA Doppler may be of specific clinical value8, 9, 10, 11. Late‐onset FGR is caused by a milder degree of placental insufficiency and is more difficult to distinguish from the constitutionally small fetus. At present, there is a lack of evidence for the optimal timing of delivery in these pregnancies. A randomized clinical trial in women with FGR at term indicated that induction of labor is not harmful19. A large national cohort study indicated that, for suspected FGR at 37 weeks, expectant management for a further week leads to less mortality than delivery, while continuation of pregnancy after 38 weeks is related to an increased risk, although this trend only reached statistical significance from 40 weeks onwards (relative risk, 2.46 (95% CI, 1.80–3.36))20.

that, for suspected FGR at 37 weeks, expectant management for a further week leads to less mortality than delivery, while continuation of pregnancy after 38 weeks is related to an increased risk, although this trend only reached statistical significance from 40 weeks onwards (relative risk, 2.46 (95% CI, 1.80–3.36))20. This review found that studies investigating CPR and MCA Doppler in late FGR are scarce, and that they evaluated different perinatal outcomes, which precluded a meta‐analysis in this specific group. For pregnancies with EFW or BW < 10th percentile at ≥ 32 weeks, nine studies investigated prognostic accuracy of the CPR and eight that of MCA Doppler. We observed a large variation in accuracy of both the CPR and MCA Doppler and, with the available data, were not able to support the theory that the CPR or MCA Doppler is of particular clinical value in late FGR. Another clinical role of CPR and MCA Doppler has been suggested in low‐risk AGA pregnancy8. In this subgroup, we observed predominantly similar prognostic accuracy to that in SGA pregnancies, although sensitivity appeared to decrease. Routinely evaluating CPR or MCA Doppler in a low‐risk population with a low prevalence of adverse outcomes would lead to a substantial number of false negatives, and therefore would leave many at‐risk fetuses undetected. Moreover, false positives could lead to unnecessary and potentially harmful interventions in this population. Unless the use of CPR or MCA Doppler is validated by clinical trials, we cannot recommend them in this setting.

substantial number of false negatives, and therefore would leave many at‐risk fetuses undetected. Moreover, false positives could lead to unnecessary and potentially harmful interventions in this population. Unless the use of CPR or MCA Doppler is validated by clinical trials, we cannot recommend them in this setting. Implications for research The reproducibility and interpretation of observational studies on CPR could be improved in the future by using set inclusion criteria, completeness in reporting and by agreeing on standard outcome measures for FGR by consensus21, 22. We were unable to analyze the incremental value of combining UA Doppler with CPR, owing to a lack of studies specifically addressing this question. This could be answered by obtaining individual participant data. The logical next step is to evaluate further the effectiveness of the CPR in clinical trials. Randomized clinical trials on a prognostic topic, such as sonographic markers, have their own challenges. The trial would be effective only if the clinician follows the result of the prognostic test under study, and the test would be beneficial only if subsequent interventions are effective23. It is reasonable to use a higher threshold for CPR, for instance the 10th or even 20th percentile, in order to allow identification of the optimal threshold. Preferably, studies should measure both UA Doppler and CPR and randomly allocate women with discordant test results (i.e. normal UA Doppler and abnormal CPR, or vice versa) to immediate delivery or continuation of pregnancy23. Only when such studies show a difference in outcome could one conclude that the CPR is of additional value to UA Doppler alone.

both UA Doppler and CPR and randomly allocate women with discordant test results (i.e. normal UA Doppler and abnormal CPR, or vice versa) to immediate delivery or continuation of pregnancy23. Only when such studies show a difference in outcome could one conclude that the CPR is of additional value to UA Doppler alone. Conclusion This review shows that calculating the CPR with MCA Doppler can improve on the accuracy of UA Doppler assessment in the prediction of adverse perinatal outcome in singleton pregnancies. This may be of particular importance in late FGR, although this could not be demonstrated with the available evidence. Clinical trials are needed to evaluate the effectiveness of the CPR for guiding clinical management in specific subgroups, such as late FGR fetuses. Supporting information Appendix S1 Search strategy implemented in electronic search of PubMed, EMBASE, the Cochrane library and ClinicalTrials.gov. Free text terms (Cochrane Library), MeSH (PubMed) and Emtree (EMBASE) controlled terms detailed. Appendix S2 Reference details of 128 studies on prognostic accuracy of umbilical artery Doppler compared to cerebroplacental ratio and/or middle cerebral artery Doppler in prediction of adverse perinatal outcome included in systematic review and meta‐analysis. Appendix S3 QUADAS‐2 quality evaluation of 128 included studies, showing risk of bias and concerns regarding applicability.

Appendix S2 Reference details of 128 studies on prognostic accuracy of umbilical artery Doppler compared to cerebroplacental ratio and/or middle cerebral artery Doppler in prediction of adverse perinatal outcome included in systematic review and meta‐analysis. Appendix S3 QUADAS‐2 quality evaluation of 128 included studies, showing risk of bias and concerns regarding applicability. Appendix S4 Thresholds of middle cerebral artery Doppler (Table A) and of cerebroplacental ratio (Table B) used in included studies for the three different Doppler indices. References for studies that assessed normal ranges are provided. Appendix S5 Reported sensitivities and specificities of middle cerebral artery Doppler (Table A) and cerebroplacental ratio (Table B) in prediction of adverse perinatal outcomes in studies investigating fetal growth restriction or small‐for‐gestational age at ≥ 32 weeks' gestation. Data are given as sensitivity (95% CI) or specificity (95% CI). Table S1 Characteristics of 128 studies on prognostic accuracy of umbilical artery Doppler compared to cerebroplacental ratio and/or middle cerebral artery Doppler in prediction of adverse perinatal outcome included in systematic review and meta‐analysis Table S2 Heterogeneity analysis P‐values of overall test performance of cerebroplacental ratio and middle cerebral artery Doppler. Fetal growth, term and timing of measurement were assessed for their impact on heterogeneity

Table S1 Characteristics of 128 studies on prognostic accuracy of umbilical artery Doppler compared to cerebroplacental ratio and/or middle cerebral artery Doppler in prediction of adverse perinatal outcome included in systematic review and meta‐analysis Table S2 Heterogeneity analysis P‐values of overall test performance of cerebroplacental ratio and middle cerebral artery Doppler. Fetal growth, term and timing of measurement were assessed for their impact on heterogeneity Figure S1 Forest plots of sensitivity and specificity of middle cerebral artery Doppler (a) and cerebroplacental ratio (b) in prediction of different adverse perinatal outcomes in 128 included studies. Click here for additional data file. ACKNOWLEDGMENTS The authors thank Katja Jordanova, Marta Jozwiak and Rui Wang for helping with translations.

INTRODUCTION The development of methods for the analysis of circulating cell‐free DNA (cfDNA) has dramatically changed prenatal screening for fetal trisomy and opened up new possibilities for other types of non‐invasive genetic testing. cfDNA originates predominantly from hematopoietic cell lines1; however, during pregnancy, a minor proportion derives from the conceptus2, 3. The proportion of pregnancy‐derived cfDNA in maternal plasma, commonly termed fetal fraction, averages 10%, but is subject to significant variation4, 5. For any kind of fetal cfDNA‐based testing, it is reasonable to assume that a minimum amount of fetal cfDNA must be present. This is supported by theoretical models demonstrating that the ability to distinguish between fetal trisomy and disomy suffers with decreasing fetal fraction6, 7. As a result, measurement of fetal fraction is now recognized widely as an essential quality metric8, 9, 10.

a minimum amount of fetal cfDNA must be present. This is supported by theoretical models demonstrating that the ability to distinguish between fetal trisomy and disomy suffers with decreasing fetal fraction6, 7. As a result, measurement of fetal fraction is now recognized widely as an essential quality metric8, 9, 10. Many laboratories measure fetal fraction, but different methodologies have necessitated different approaches. Quantitation of Y‐chromosome sequences was the first method employed because this DNA can be taken as a direct representation of fetal cfDNA, but is informative only in pregnancies with a male fetus4, 11. Analysis of single‐nucleotide polymorphisms (SNPs) is more powerful because it can be applied regardless of fetal sex12. It is well suited to methodologies that target specific sequences but is difficult to apply to massively parallel shotgun sequencing approaches, as these tend to have a very shallow depth at any given polymorphic locus. Massively parallel shotgun sequencing‐based methodologies have therefore relied on less direct assessment based on observed differences between fetal and maternal cfDNA fragments13, 14, 15. There is, however, concern about the consistency of the different approaches16.

very shallow depth at any given polymorphic locus. Massively parallel shotgun sequencing‐based methodologies have therefore relied on less direct assessment based on observed differences between fetal and maternal cfDNA fragments13, 14, 15. There is, however, concern about the consistency of the different approaches16. We have previously described the application of Digital ANalysis of Selected Regions (DANSR) to fetal trisomy screening17, 18. This includes non‐polymorphic assays for the determination of chromosome proportions and polymorphic assays leveraging SNPs for fetal‐fraction determination in a single reaction12. The current study aimed to evaluate rigorously this method for fetal‐fraction estimation using a number of parameters. First, accuracy was assessed by comparing DANSR assays with quantitation of Y‐chromosome sequences. Second, reproducibility was determined in a series of paired tubes drawn from the same patient. Finally, we evaluated whether this method is equally informative across populations.

stimation using a number of parameters. First, accuracy was assessed by comparing DANSR assays with quantitation of Y‐chromosome sequences. Second, reproducibility was determined in a series of paired tubes drawn from the same patient. Finally, we evaluated whether this method is equally informative across populations. METHODS Dataset The data analyzed in this study were sourced from existing information generated as part of clinical testing of venous blood samples submitted for the Harmony® prenatal test to the College of American Pathologists‐accredited and Clinical Laboratory Improvement Amendments‐certified Ariosa Diagnostics laboratory in San Jose, CA, USA. Patients provided consent for the performed testing and samples were processed in accordance with all applicable laws. All data were anonymized prior to analysis. Analysis was limited to samples from singleton pregnancies of at least 10 weeks' gestation, as reported by ordering providers on the sample submission form. Samples with insufficient fetal fraction for analysis (less than 4%, as measured using DANSR assays) and samples that did not meet laboratory quality‐control thresholds were excluded. Samples were included irrespective of the probability score generated for fetal aneuploidy.

ing providers on the sample submission form. Samples with insufficient fetal fraction for analysis (less than 4%, as measured using DANSR assays) and samples that did not meet laboratory quality‐control thresholds were excluded. Samples were included irrespective of the probability score generated for fetal aneuploidy. Sample collection and test method Sample preparation and analysis for the Harmony prenatal test were performed as described previously12, 18. In brief, blood samples were collected in either cfDNA BCT tubes (Streck, Omaha, NE, USA) or cfD tubes (Roche, Pleasanton, CA, USA) and processed to yield cell‐free plasma within 7 days of collection. cfDNA was purified from this plasma and DANSR products were made using non‐polymorphic assays of chromosomes 13, 18, 21, X and Y and polymorphic assays of chromosomes 1–12. DANSR products were hybridized and signals measured using a custom microarray. Fetal‐fraction measurement Fetal‐fraction measurement was performed, as described previously, using 576 DANSR assays designed against polymorphic loci on chromosomes 1–12 (Figure 1)12, 18. The assays are unbiased, cover each chromosome uniformly and target SNPs with high minor‐allele frequencies in the HapMap dataset19 to maximize the number of informative loci for fetal‐fraction measurement. Figure 1 Distribution of minor‐allele frequencies (MAFs) of Harmony® prenatal test polymorphic assays across five major populations in the 1000 Genomes Project, with mean MAF () and expected informative fraction (IF) provided for each population.

Fetal‐fraction measurement Fetal‐fraction measurement was performed, as described previously, using 576 DANSR assays designed against polymorphic loci on chromosomes 1–12 (Figure 1)12, 18. The assays are unbiased, cover each chromosome uniformly and target SNPs with high minor‐allele frequencies in the HapMap dataset19 to maximize the number of informative loci for fetal‐fraction measurement. Figure 1 Distribution of minor‐allele frequencies (MAFs) of Harmony® prenatal test polymorphic assays across five major populations in the 1000 Genomes Project, with mean MAF () and expected informative fraction (IF) provided for each population. UOG-19036-FIG-0001-bRelative allele frequencies were used to compute fetal fraction with the minor percentage of the fraction assumed to be fetal. No prior genotyping of the mother, father or fetus was carried out. The relative signal intensities of DANSR assays corresponding to non‐polymorphic loci were evaluated to estimate the relative amount of each of the interrogated chromosomes. The relative signal intensities for the Y chromosome were taken as a direct representation of fetal fraction in pregnancies with a putative male fetus. DANSR single‐nucleotide polymorphism assays vs Y‐chromosome quantitation Fetal fraction was determined by both methods in a consecutive series of 47 512 plasma samples from singleton pregnancies with a putative male fetus. Pearson's correlation coefficient (r) was calculated to determine the correlation between methods of fetal‐fraction measurement.

phism assays vs Y‐chromosome quantitation Fetal fraction was determined by both methods in a consecutive series of 47 512 plasma samples from singleton pregnancies with a putative male fetus. Pearson's correlation coefficient (r) was calculated to determine the correlation between methods of fetal‐fraction measurement. Fetal‐fraction reproducibility DANSR SNP assays were used to measure fetal fraction in two tubes from 734 randomly chosen, de‐identified, frozen plasma samples. Patients provided consent to have their sample de‐identified and used for additional study. Over a period spanning 3 months, the first and second tubes were processed separately from one another but together with other clinical samples. Second‐tube fetal‐fraction information was compared with anonymized first‐tube data. Pearson's correlation (r) and the median absolute deviation from the mean were calculated. Calculation of fraction of informative loci DANSR assays are designed to be equally informative across global populations. Figure 1 shows mean minor‐allele frequencies and the predicted fraction of informative loci for different populations based on data from the 1000 Genomes Project20. To assess experimentally whether DANSR assays are informative for different populations, the number of informative loci was determined in 13 988 clinical samples consisting of the reproducibility dataset and other samples analyzed in the same time frame. The fraction of informative loci is calculated as the proportion of determined maternal homozygous loci that are determined to be heterozygous in the fetus.

e number of informative loci was determined in 13 988 clinical samples consisting of the reproducibility dataset and other samples analyzed in the same time frame. The fraction of informative loci is calculated as the proportion of determined maternal homozygous loci that are determined to be heterozygous in the fetus. RESULTS The correlation between fetal fractions measured using DANSR SNP assays and relative Y‐chromosome quantitation for 47 512 samples in which Y‐chromosome sequences were present is shown in Figure 2. There was a strong correlation between the two methods (r = 0.97). Figure 2 Correlation between fetal fraction determined using polymorphic Digital ANalysis of Selected Regions (DANSR) assays and that determined using Y‐sequence quantitation (r = 0.97) in 50 000 samples from women with singleton pregnancy and putative male fetus. Identity and best‐fit lines () are superimposed. UOG-19036-FIG-0002-bFigure 3 depicts the reproducibility of fetal‐fraction measurement for 734 patient samples in two tubes drawn at the same time. The correlation (r) between measurements was 0.98, with a median absolute deviation from the mean of 0.26%. Figure 3 Correlation between first‐ and second‐tube fetal‐fraction measurements (r = 0.98) in 734 patient samples from women with singleton pregnancy. Identity () and best‐fit () lines are shown.

UOG-19036-FIG-0002-bFigure 3 depicts the reproducibility of fetal‐fraction measurement for 734 patient samples in two tubes drawn at the same time. The correlation (r) between measurements was 0.98, with a median absolute deviation from the mean of 0.26%. Figure 3 Correlation between first‐ and second‐tube fetal‐fraction measurements (r = 0.98) in 734 patient samples from women with singleton pregnancy. Identity () and best‐fit () lines are shown. UOG-19036-FIG-0003-bThe samples received by the Ariosa Diagnostics laboratory originate from more than 100 countries and territories across five continents and are therefore representative of a range of global racial diversity. In 13 988 clinical samples, the median number of observed fractions of informative loci associated with homozygous maternal genotype was 0.42 (SD, 0.041) (Figure 4), which is consistent with a mean worldwide minor‐allele frequency of 38%. Figure 4 Distribution of observed fraction of putative informative loci in 13 988 samples from women with singleton pregnancy, showing mean informative fraction (0.42 ± 0.041, ), which is consistent with underlying mean minor‐allele frequency in Digital ANalysis of Selected Regions (DANSR) polymorphic assays across worldwide populations (0.38).

bserved fraction of putative informative loci in 13 988 samples from women with singleton pregnancy, showing mean informative fraction (0.42 ± 0.041, ), which is consistent with underlying mean minor‐allele frequency in Digital ANalysis of Selected Regions (DANSR) polymorphic assays across worldwide populations (0.38). UOG-19036-FIG-0004-bDISCUSSION Despite the increasing recognition of fetal fraction as an important quality metric in cfDNA testing for fetal trisomy, there has been little focus on the variety of methods employed and the need to ensure their reliability. Underestimating fetal fraction may cause valid samples to be rejected unnecessarily. Overestimating fetal fraction may lead to samples with insufficient fetal cfDNA being tested and a risk of false‐negative results6, 7. This may not be easily discernible in clinical studies designed solely to evaluate test performance characteristics, such as sensitivity and specificity, due to the number of affected pregnancies studied at lower fetal fractions, but can be addressed by direct investigation.

ed and a risk of false‐negative results6, 7. This may not be easily discernible in clinical studies designed solely to evaluate test performance characteristics, such as sensitivity and specificity, due to the number of affected pregnancies studied at lower fetal fractions, but can be addressed by direct investigation. Fetal‐fraction measurement using relative competitive quantitation at polymorphic loci with microarray allows for a single laboratory workflow to perform simultaneously fetal‐fraction measurement and sequence quantitation for trisomy assessment in the same assay under the same conditions. The current study represents a large‐scale assessment of polymorphic assays, as used to measure fetal fraction as part of the Harmony prenatal test. The accuracy of fetal‐fraction measurement was demonstrated by comparison with measuring Y‐sequence signal intensities in a cohort of over 47 000 clinical samples. The fetal‐fraction calculations are also consistent with little variation between paired samples of two tubes drawn from the same patient at the same time. The correlation of fetal‐fraction measurement between the two tubes was high, with an average deviation of ±0.26%. In practical terms, fetal‐fraction measurements in samples with a theoretical absolute fetal fraction of 7% would be expected to vary on average by between only 6.74% and 7.26%. Finally, the SNPs chosen are informative across global populations, an important requirement given the rapid uptake of cfDNA screening worldwide.

practical terms, fetal‐fraction measurements in samples with a theoretical absolute fetal fraction of 7% would be expected to vary on average by between only 6.74% and 7.26%. Finally, the SNPs chosen are informative across global populations, an important requirement given the rapid uptake of cfDNA screening worldwide. Until now, most published studies addressing fetal‐fraction estimation have been limited to proof‐of‐concept studies13, 14, 15, 21, 22. Many of these have centered on the generation of models to correlate fetal fraction with the observed differences between maternal and fetal cfDNA, such as differential methylation at specific regions13, distribution of cfDNA fragment size14, patterns of DNA sequence representation15, nucleosome positioning21 and preferred sequence end sites22. The differences are likely to be the consequence of differences in chromatin structure and transcriptional processes in maternal leukocytes compared with placental trophoblasts; however, the underlying biology is yet to be elucidated fully22. Although some are being used clinically15, these models are complex, with tens of thousands of historical parameters to scale the sequencing count from each chromosome bin and there is considerable overlap between maternal and fetal cfDNA in the studied attributes. Furthermore, the stability of these differences may be a potential concern for use in fetal‐fraction estimation. The possible influence of factors, such as gestational age and maternal elements, has not to our knowledge been evaluated comprehensively in larger series. In one recently published study, van Beek et al.23 directly compared six methods in a series of 375 pregnancies with a male fetus and found that methods based on binned autosomal read counts15 and nucleosome positioning21 did not perform reliably as compared with Y‐based methods23.

ted comprehensively in larger series. In one recently published study, van Beek et al.23 directly compared six methods in a series of 375 pregnancies with a male fetus and found that methods based on binned autosomal read counts15 and nucleosome positioning21 did not perform reliably as compared with Y‐based methods23. A challenge for the evaluation of any assay that estimates fetal fraction is the establishment of a reference against which to compare the calculated proportions. Quantitation via Y‐chromosome sequences is therefore considered to be a gold standard with the least potential for confounding factors and was applied in the current study to a large cohort of real pregnancy samples with male fetuses that were the same processed samples and data sources as those used for the assessment of trisomy. This study was also constrained by the use of a clinical laboratory platform that limits the reporting of fetal fraction to samples meeting a minimum requirement of 4% fetal fraction. We were not able to determine the lowest reasonable threshold for fetal‐fraction measurement. Just as different methods for trisomy screening are well validated before clinical implementation, so should be assay designs for fetal‐fraction measurement. This study provides a useful benchmark for ensuring reliability. Evaluation would include demonstration of both accuracy and reproducibility. Moreover, ensuring that the used method performs well across all pregnancies is warranted. ACKNOWLEDGMENT Funding for this study was supplied by Roche Sequencing Solutions Inc.

INTRODUCTION The levator ani muscles provide support for the pelvic organs. During vaginal delivery, trauma to these muscles will occur in one‐fifth to one‐third of women1, 2. This trauma weakens pelvic floor support and may cause problems such as pelvic organ prolapse and urinary incontinence3, 4. During delivery, the part of the levator ani muscle that surrounds the urogenital hiatus, the puborectalis muscle (PRM), has to stretch to more than twice its original length and may therefore sustain damage5, 6. It is not well understood why some women experience pelvic floor problems after delivery while others do not. Some studies suggest that it might be due to changes in muscle structure that occur during pregnancy, prior to delivery7, 8, 9. It is to be expected that these changes will be reflected in muscle functionality. Recently, in studies that used three‐ and four‐dimensional (3D/4D) ultrasound imaging to visualize the PRM, new informative parameters of the PRM were obtained; for example, hiatal dimensions10, 11, mean echogenicity of the PRM (MEP)9, 12, 13 and strain14. Hiatal dimensions and strain are related to, and provide insight into the functionality of, the PRM. Early pregnancy MEP has been shown to be related to mode of delivery13.

ew informative parameters of the PRM were obtained; for example, hiatal dimensions10, 11, mean echogenicity of the PRM (MEP)9, 12, 13 and strain14. Hiatal dimensions and strain are related to, and provide insight into the functionality of, the PRM. Early pregnancy MEP has been shown to be related to mode of delivery13. However, in these studies parameters of the PRM were obtained manually, which is time‐consuming, hindering their introduction into clinical practice. Sindhwani et al.15 proposed a method to automate the measurement of hiatal dimensions. However, these are static two‐dimensional (2D) measurements, not exploiting the full potential of 3D/4D ultrasound. Therefore, the aim of this work was to develop an automatic segmentation method for the PRM in 3D transperineal ultrasound images. Active appearance models (AAMs), which require manually annotated training data, have proved to be reliable in automatically segmenting structures in 3D ultrasound (e.g. the left ventricle16). As 3D segmentation of the PRM on 3D ultrasound is, to the best of our knowledge, as yet unexplored, we present a manual 3D segmentation protocol for the PRM on 3D transperineal ultrasound. We assessed its reproducibility and used it to generate training data for an AAM. The performance of the fully automatic AAM was then analyzed by comparing it with manual segmentation.

the best of our knowledge, as yet unexplored, we present a manual 3D segmentation protocol for the PRM on 3D transperineal ultrasound. We assessed its reproducibility and used it to generate training data for an AAM. The performance of the fully automatic AAM was then analyzed by comparing it with manual segmentation. METHODS Data The data for this study were obtained from a dataset acquired by van Veelen et al.17 that consists of 3D/4D transperineal ultrasound data of the pelvic floor from 280 nulliparous women with a singleton pregnancy who were scanned at multiple time points pre‐ and postpartum. In the current study we focused on scans acquired at 12 weeks' gestation. The acquisition was performed using a GE Voluson 730 Expert system (GE Medical Systems, Zipf, Austria), equipped with a RAB 4–8‐MHz curved‐array volume transducer. The settings of the system were kept constant to minimize variation across acquisitions, since the main aim of this study was to investigate the MEP. The acquisition started with the PRM at rest, and then the women were asked to contract their PRM fully and, afterwards, to perform a Valsalva maneuver, which results in a stretched PRM.

the system were kept constant to minimize variation across acquisitions, since the main aim of this study was to investigate the MEP. The acquisition started with the PRM at rest, and then the women were asked to contract their PRM fully and, afterwards, to perform a Valsalva maneuver, which results in a stretched PRM. A dataset of 63 videoclips was selected randomly from the original dataset. While selecting, videoclips were checked by an observer for the following inclusion criteria: the PRM is contained entirely in the field of view; the symphysis is in the field of view to allow measurement of the minimal hiatal dimensions; and image quality is sufficient. Owing to the consistent ultrasound settings, some images had poor contrast, insufficient for 3D delineation of the PRM. For the quality criterion, two observers had to agree on whether image quality was sufficient for inclusion. Thirteen videoclips did not meet all the criteria and were discarded. Of these, five did not fully capture the PRM in 3D, five did not capture the symphysis and three were of poor image quality, thus 50 videoclips were included in the study. From each of these, one frame in which the PRM was at rest was selected. A randomly selected subset of data from 20 subjects was used to analyze the manual segmentations.

fully capture the PRM in 3D, five did not capture the symphysis and three were of poor image quality, thus 50 videoclips were included in the study. From each of these, one frame in which the PRM was at rest was selected. A randomly selected subset of data from 20 subjects was used to analyze the manual segmentations. The ultrasound clips were converted to DICOM in 4D View 9.0 (GE Medical Systems). In‐house developed software, based on MeVisLab 2.6.2 (MeVis Medical Solutions, Bremen, Germany)18, was used for selecting the frame with the PRM at rest, manual segmentation using splines, AAM training and AAM matching. 3D segmentation protocol The 3D segmentation protocol is an extension of an existing validated protocol for 2D segmentation11, 12. To ensure optimal visibility of the PRM, the protocol starts with rotating to the slice with minimal hiatal dimensions (SMHD) (Figure 1c)10, which allows browsing through the tomographic ultrasound images19. Figure 1 Manual three‐dimensional segmentation on transperineal ultrasound of puborectalis muscle (PRM) (delineated) in different axial slices, from caudal to cranial, showing: (a) caudal limit of PRM; (b) slice in which PRM is disconnected visually from pubic symphysis; (c) slice of minimal hiatal dimensions; (d) slice in which posterior part of PRM is out‐of‐plane; (e) cranial limit of PRM; (f) segmentation of PRM in midsagittal slice, with green line showing how to determine boundary between external anal sphincter and PRM; and (g) position of slices (a)–(e) on midsagittal slice.

s; (c) slice of minimal hiatal dimensions; (d) slice in which posterior part of PRM is out‐of‐plane; (e) cranial limit of PRM; (f) segmentation of PRM in midsagittal slice, with green line showing how to determine boundary between external anal sphincter and PRM; and (g) position of slices (a)–(e) on midsagittal slice. UOG-18927-FIG-0001-c2D segmentation of the PRM in the SMHD has been presented previously12, and is shown in Figure 1c. In slices close to the SMHD, the segmented shape of the PRM is the same as in the SMHD; however, the segmented shape changes in slices further away. Caudal from the SMHD, the segmented shape of the PRM is detached visually from the symphysis, as shown in Figure 1b. The distance between the segmented area and the symphysis will increase further, moving to the caudal limit of the PRM. This caudal limit (Figure 1a) is found by using the midsagittal plane (Figure 1f), in which the distinction between the PRM and the external sphincter is much clearer than in the axial view. Cranial from the SMHD, the posterior part of the PRM will disappear (Figure 1d). So from here, the PRM needs to be segmented as two separate parts. It is hard to determine the cranial boundary of the PRM. However, Singh et al.20 showed that there is sharp angulation between the PRM and the iliococcygeus muscle. Therefore, the last cranial slice is the one before the PRM becomes indistinguishable from its surroundings or before the areas that appear to be part of the PRM start moving outwards (Figure 1e).

ary of the PRM. However, Singh et al.20 showed that there is sharp angulation between the PRM and the iliococcygeus muscle. Therefore, the last cranial slice is the one before the PRM becomes indistinguishable from its surroundings or before the areas that appear to be part of the PRM start moving outwards (Figure 1e). Active appearance model AAMs model the typical variation in shape and texture of an object presented in training data21. When the manual PRM data are used to train the model, it learns the natural variability in appearance of the PRM in 3D transperineal ultrasound data. In new data (not present in the training set), this model can be used to predict the position of the PRM. We used the AAM implementation described by van Stralen et al.16, with the leave‐one‐out method to train and match (finding the PRM) the AAMs, to avoid training bias in their evaluation. This means that we created 50 AAMs, using 49 manually segmented ultrasound images for the training of one AAM and the last ultrasound image for matching. Therefore, we could optimally use the manual segmentation data, both for training and as ground truth to analyze the performance of automatic segmentation.

valuation. This means that we created 50 AAMs, using 49 manually segmented ultrasound images for the training of one AAM and the last ultrasound image for matching. Therefore, we could optimally use the manual segmentation data, both for training and as ground truth to analyze the performance of automatic segmentation. Validation To validate the manual segmentation protocol, a randomly selected subset of 20 subjects had their ultrasound PRM segmented by a second observer as well as by the first observer for a second time more than a month later. This allowed for analysis of inter‐ and intraobserver performance. The performances of manual and automated segmentation were compared. Statistical analysis was performed using SPSS v. 23 (SPSS Inc., Chicago, IL, USA) and Excel 2011 (Microsoft Office, Microsoft Corp., Redmond, WA, USA). Means, SDs and intraclass correlation coefficients (ICCs) with their 95% CIs were used to compare MEP and volume acquired by observers and computer. ICC results were classified according to the subgroups defined by Landis and Koch22.

hicago, IL, USA) and Excel 2011 (Microsoft Office, Microsoft Corp., Redmond, WA, USA). Means, SDs and intraclass correlation coefficients (ICCs) with their 95% CIs were used to compare MEP and volume acquired by observers and computer. ICC results were classified according to the subgroups defined by Landis and Koch22. Although volume and MEP measurements are relevant parameters with which to analyze the performance of manual and automatic segmentation, good agreement of these does not necessarily mean good agreement in the positions of two segmentations. Therefore, we used MeVisLab to calculate three other parameters that better reflect agreement between segmentations; these were mean absolute distance (MAD), Hausdorff distance and Dice coefficient (D). The absolute distance is the minimum distance from one point on the segmentation to a point on the surface of the other segmentation; an example is given in Figure 2. MAD is the mean of all absolute distance measurements between two segmentations, showing on average how far apart from each other the two surfaces are. The Hausdorff distance is the maximum absolute distance between two segmentations; this gives insight into the maximum error made. D measures the overlap between segmentations, and is calculated as: D=2X∩YX+Y Figure 2 Automatic (red) and manual (blue) segmentation of puborectalis muscle in slice with minimal hiatal dimensions. Arrows indicate one measurement of absolute distance between two segmentations. Green area is overlap between segmentations.

Although volume and MEP measurements are relevant parameters with which to analyze the performance of manual and automatic segmentation, good agreement of these does not necessarily mean good agreement in the positions of two segmentations. Therefore, we used MeVisLab to calculate three other parameters that better reflect agreement between segmentations; these were mean absolute distance (MAD), Hausdorff distance and Dice coefficient (D). The absolute distance is the minimum distance from one point on the segmentation to a point on the surface of the other segmentation; an example is given in Figure 2. MAD is the mean of all absolute distance measurements between two segmentations, showing on average how far apart from each other the two surfaces are. The Hausdorff distance is the maximum absolute distance between two segmentations; this gives insight into the maximum error made. D measures the overlap between segmentations, and is calculated as: D=2X∩YX+Y Figure 2 Automatic (red) and manual (blue) segmentation of puborectalis muscle in slice with minimal hiatal dimensions. Arrows indicate one measurement of absolute distance between two segmentations. Green area is overlap between segmentations. UOG-18927-FIG-0002-cwhere |X ∩ Y| is the volume of the overlap (depicted in Figure 2 as the green area) and X and Y are the volumes of the segmentations that are being compared. If there is no overlap between segmentations, D is 0; if there is complete overlap, D is 1. Good agreement between segmentations is reflected in low MAD and Hausdorff values and in high D values.

e overlap (depicted in Figure 2 as the green area) and X and Y are the volumes of the segmentations that are being compared. If there is no overlap between segmentations, D is 0; if there is complete overlap, D is 1. Good agreement between segmentations is reflected in low MAD and Hausdorff values and in high D values. RESULTS The resulting volume of a manual segmentation is shown in Figure 3. In Table 1, mean (SD) and ICCs for MEP and volume for automatic vs manual segmentations, and inter‐ and intraobserver manual segmentations, are presented. ICCs for comparison of automatic and manual segmentations for MEP and volume were, respectively, very good and good. Inter‐ and intraobserver ICCs for manual segmentation showed very good agreement for both volume and MEP. Figure 3 Example of manual three‐dimensional segmentation in transperineal ultrasound of puborectalis muscle, shown from different angles, in sagittal (a), coronal (b) and axial (c) slices. UOG-18927-FIG-0003-cTable 1 Mean echogenicity (MEP) and volume of transperineal ultrasound three‐dimensional segmentation of puborectalis muscle performed manually (by observer) vs automatically (by computer), manually by two independent observers and manually twice by same observer ≥ 1 month apart, with corresponding intraclass correlation coefficients (ICCs)

ity (MEP) and volume of transperineal ultrasound three‐dimensional segmentation of puborectalis muscle performed manually (by observer) vs automatically (by computer), manually by two independent observers and manually twice by same observer ≥ 1 month apart, with corresponding intraclass correlation coefficients (ICCs) Segmentation method (n = 45) MEP (a.u.) ICC (95% CI) Volume (mL) ICC (95% CI) Observer 148 ± 16 0.968 (0.941–0.982) 9.4 ± 1.8 0.626 (0.327–0.794) Computer 147 ± 17 9.8 ± 2.6 Interobserver (n = 20) Observer 1 151 ± 20 0.987 (0.962–0.995) 8.5 ± 1.7 0.910 (0.771–0.964) Observer 2 153 ± 18 8.6 ± 2.1 Intraobserver (n = 20) First set of observations 151 ± 20 0.991 (0.978–0.996) 8.5 ± 1.7 0.877 (0.694–0.951) Second set of observations 152 ± 19 8.8 ± 1.7 MEP and volume are given as mean ± SD. a.u., arbitrary units. In Figure 4, D, MAD and Hausdorff distance are presented in box‐and‐whisker plots. The parameters that determine the distance between two segmentations, Hausdorff distance and MAD, correspond to only a few voxels mismatch. The D‐values are moderate. For automatic segmentation, five mismatches were identified based on high MAD and/or low D; these appear as outliers in Figure 4. These subjects were not included in the ICC analysis in Table 1, since measuring MEP and volume is relevant only when automatic segmentation is successful. Hausdorff distance, MAD and D show that automatic segmentation is comparable with manual segmentation if the mismatches are ignored.

appear as outliers in Figure 4. These subjects were not included in the ICC analysis in Table 1, since measuring MEP and volume is relevant only when automatic segmentation is successful. Hausdorff distance, MAD and D show that automatic segmentation is comparable with manual segmentation if the mismatches are ignored. Figure 4 Box‐and‐whisker plots of Dice coefficient (a), mean absolute distance (b) and Hausdorff distance (c) for computer (Comp)‐ vs observer (Obs)‐derived three‐dimensional segmentation of puborectalis muscle (n = 50) and inter‐ and intraobserver manual segmentations (n = 20). Boxes with internal lines represent median and interquartile range (IQR), whiskers are range excluding outliers more than 1.5 × IQR from upper and lower quartile, and + are outliers. UOG-18927-FIG-0004-bIn Figure 5, the mean of all manual segmentations is visualized, showing average absolute distance between the manual segmentations and 45 successful automatic ones. This shows which areas are the most complicated in segmentation. These areas are the cranial and caudal limits of the PRM. Also, the site of attachment to the symphysis was found to have a high distance between manual and automatic segmentations. Figure 5 Mean of manual three‐dimensional segmentations of puborectalis muscle, color‐coded according to average absolute distances between manual and successful automatic segmentations (n = 45).

UOG-18927-FIG-0004-bIn Figure 5, the mean of all manual segmentations is visualized, showing average absolute distance between the manual segmentations and 45 successful automatic ones. This shows which areas are the most complicated in segmentation. These areas are the cranial and caudal limits of the PRM. Also, the site of attachment to the symphysis was found to have a high distance between manual and automatic segmentations. Figure 5 Mean of manual three‐dimensional segmentations of puborectalis muscle, color‐coded according to average absolute distances between manual and successful automatic segmentations (n = 45). UOG-18927-FIG-0005-cDISCUSSION To measure 3D and 4D clinical parameters of the PRM on transperineal 3D/4D ultrasound, automatic segmentation of the PRM is needed. In this study, we present a reproducible manual segmentation protocol. AAMs trained using segmentations obtained using this protocol provided promising results for automatic segmentation, with results comparable with those of manual segmentation.

nsperineal 3D/4D ultrasound, automatic segmentation of the PRM is needed. In this study, we present a reproducible manual segmentation protocol. AAMs trained using segmentations obtained using this protocol provided promising results for automatic segmentation, with results comparable with those of manual segmentation. Since this is the first attempt at manual segmentation of the PRM on 3D ultrasound, the results can be compared only with segmentation results of magnetic resonance imaging or cadaver studies. The segmentation results of the PRM obtained in such studies5, 23, 24, 25, 26, 27 show very similar shapes to the those obtained in the current study, an example of which is shown in Figure 3. However, some of these studies5, 24, 26 show that there is a thin layer of muscle fibers from other pelvic floor muscles running at the hiatal side of the PRM. These fibers are also seen in endovaginal ultrasound studies28, 29. However, using endovaginal ultrasound, one is able to scan at higher frequencies to visualize the PRM, since it lies closer to the probe. On transperineal ultrasound, this layer of muscle fiber cannot be distinguished from the PRM. This does not influence the performance of automatic segmentation, but it may be of influence if this manual segmentation protocol is used for functional investigation of the PRM.

ze the PRM, since it lies closer to the probe. On transperineal ultrasound, this layer of muscle fiber cannot be distinguished from the PRM. This does not influence the performance of automatic segmentation, but it may be of influence if this manual segmentation protocol is used for functional investigation of the PRM. The main reason for the AAM mismatches is feces in the rectum, in which case the PRM appears darker and the feces white and the algorithm therefore segments the feces instead of the PRM. However, in 13 of the 50 images, of which only four resulted in a mismatch using the algorithm, feces can be clearly seen in the rectum. The last mismatch case was of a PRM that appeared very dark compared with others in the training dataset. It is probable that these problems will be solved with more training data.

M. However, in 13 of the 50 images, of which only four resulted in a mismatch using the algorithm, feces can be clearly seen in the rectum. The last mismatch case was of a PRM that appeared very dark compared with others in the training dataset. It is probable that these problems will be solved with more training data. Hausdorff distances and MADs are comparable with the data presented by Sindhwani et al.15 on 2D segmentation of the urogenital hiatus. The 3D‐MEP ICCs are comparable with the results for 2D MEP found by Grob et al.12. The D‐coefficient, however, is much lower than the values reported for 2D segmentation of the urogenital hiatus15. The first explanation for this can be found, on comparison of the 2D and 3D cases, by approximating the volume of the PRM as a cylinder (V = πr 2h) and the area of the longitudinal cross‐section cylinder plane as A = 2rh, where r is radius and h is length. This shows that a mismatch in r has a larger influence on volume than on area. The second explanation is the shape of the PRM. As it is a very long and thin structure, a small shift has a much larger impact on the overlap than, for example, in the case of a spherical structure (or a 2D circle approximating the hiatus).

This shows that a mismatch in r has a larger influence on volume than on area. The second explanation is the shape of the PRM. As it is a very long and thin structure, a small shift has a much larger impact on the overlap than, for example, in the case of a spherical structure (or a 2D circle approximating the hiatus). The strength of this study is that it provides for the first time reliable manual and automatic 3D segmentations of the PRM. This allows for measurements of MEP and volume as clinical parameters, and might be a starting point for further functional analysis. The segmentation method is based on methods that have already been proved to be reliable in 2D10, 11, 12. Since the data were obtained using a relatively old ultrasound system, even better results may be expected from data obtained using newer systems. The data were acquired using the same settings in order to measure reliably the MEP; this may have caused suboptimal image quality in certain individual cases. The application of automated 3D segmentation automates the measurement of MEP, a clinically valuable parameter13. Using the full potential of the 3D/4D nature of the data, automating the measurements in 3D will likely improve reproducibility and sensitivity, as has been shown for quantitative analyses in other domains (e.g. 2D vs 3D functional parameters in cardiac imaging30). Moreover, more complicated parameters, like volume, shape and function of the PRM, are now open to investigation.

utomating the measurements in 3D will likely improve reproducibility and sensitivity, as has been shown for quantitative analyses in other domains (e.g. 2D vs 3D functional parameters in cardiac imaging30). Moreover, more complicated parameters, like volume, shape and function of the PRM, are now open to investigation. Anatomical validation of our manual segmentation protocol is ongoing and further research, e.g. a cadaver study, may help us to improve the segmentation protocol. Both automatic and manual segmentations were performed on data from subjects with an intact pelvic floor (nulliparous and at 12 weeks' gestation). The described method may perform suboptimally for data of patients with PRM trauma, since their data will have a different appearance and are not presented in the training data. The segmentation protocol may need to be updated and the training dataset of the AAM expanded with patient data. Currently, it is not possible to determine automatically if automatic segmentation was successful, e.g. for quality assurance, without the use of manual segmentation. However, in clinical practice, the physician can be the one to determine the success of automatic segmentation.

et of the AAM expanded with patient data. Currently, it is not possible to determine automatically if automatic segmentation was successful, e.g. for quality assurance, without the use of manual segmentation. However, in clinical practice, the physician can be the one to determine the success of automatic segmentation. In conclusion, this study presents a reliable manual segmentation protocol for the PRM on transperineal 3D/4D ultrasound of an intact pelvic floor. Furthermore, it presents automatic segmentation of the PRM based on these data, which has results comparable with those of manual segmentation; it also allows for reliable measurement of MEP. Further studies using this method may improve our understanding of the structure and (dys)‐function of the PRM.

INTRODUCTION Patients with pulmonary atresia with intact ventricular septum (PAIVS) or critical pulmonary stenosis (CPS) carry a significant risk of morbidity and mortality1. During fetal life, progression of CPS and development of PAIVS have been reported2, 3. In the absence of high‐grade tricuspid regurgitation (TR) this lesion leads to a hypoplastic right ventricle (RV), which may preclude a biventricular circulation after birth1, 4, 5. Even if biventricular repair can be achieved postnatally, RV function may remain abnormal and the potential of postnatal RV growth is limited6, 7. Therefore, the goal of fetal pulmonary valvuloplasty is to stimulate and promote prenatal RV growth in order to avoid significant RV hypoplasia at birth. It has been shown that the fetal myocardium responds to increased pre‐ and afterload with hyperplasia, in contrast to hypertrophy after birth. Therefore, prenatal treatment could be a unique opportunity to take advantage of a better ventricular growth than that achievable postnatally. A few case reports and small case series have shown that fetal pulmonary valvuloplasty is technically feasible and associated with continued growth of RV structures8, 9, 10, 11, 12, 13. The International Fetal Cardiac Intervention Registry reported 16 fetal pulmonary valvuloplasties, of which 11 procedures were successful and resulted in seven liveborn patients, five of which were discharged with a biventricular circulation. However, no data on preprocedure RV dimensions or postprocedure RV growth were recorded in these reports13.

c Intervention Registry reported 16 fetal pulmonary valvuloplasties, of which 11 procedures were successful and resulted in seven liveborn patients, five of which were discharged with a biventricular circulation. However, no data on preprocedure RV dimensions or postprocedure RV growth were recorded in these reports13. The aim of this study was to assess the immediate effects of fetal pulmonary valvuloplasty on RV size and function as well as in‐utero RV growth and postnatal outcome in fetuses with PAIVS or CPS. METHODS All patients who underwent fetal pulmonary valvuloplasty at our center between October 2000 and July 2017 were included in this study. Criteria for fetal pulmonary valvuloplasty were either membranous atresia or a critical stenosis of the pulmonary valve (PV) with a recognizable RV outflow tract (RV‐OT), retrograde flow in the ductus arteriosus and a hypoplastic hypertrophic RV with suprasystemic pressures as assessed by the velocity of the TR jet. Exclusion criteria were muscular atresia of the RV‐OT, severe TR with low velocity (< 2.5 m/s) or the presence of large RV sinusoids. The study was approved by our local ethics committee (study number K‐104‐16) and informed consent was not required.

ith suprasystemic pressures as assessed by the velocity of the TR jet. Exclusion criteria were muscular atresia of the RV‐OT, severe TR with low velocity (< 2.5 m/s) or the presence of large RV sinusoids. The study was approved by our local ethics committee (study number K‐104‐16) and informed consent was not required. All patients had an echocardiographic examination (Vivid 7®, Vivid E9®, Vivid E95®; GE Medical Systems, Zipf, Austria) a few days before and after the procedure (median interval after intervention, 1 (range, 1–3) days) by the same experienced echocardiographer (G.T.). Two‐dimensional and Doppler echocardiographic data were recorded and stored on video tape (prior to 2010) or in digital format (Echopac®, GE Medical Systems). Echocardiographic data were analyzed retrospectively for ventricular and valvular dimensions and ratios, as well as for RV filling time (duration of tricuspid valve (TV) inflow/cardiac cycle length), TV velocity time integral (TV‐VTI) × heart rate (HR), and TR velocity.

inal circulation was still unclear. Eight patients were delivered and treated in our center, whereas all other patients were delivered and managed in their respective home countries. For the longitudinal assessment of RV growth, only measurements from fetuses that were delivered and followed up at our center were used. To assess the effect of fetal pulmonary valvuloplasty on outcome, patients were grouped retrospectively according to the scoring system published by Roman et al.15. This scoring system is based on four parameters (TV/mitral valve annular diameter (TV/MV) ratio < 0.7, RV/left ventricle length (RV/LV) ratio < 0.6, RV filling time < 31.5% of cardiac cycle length and presence of RV sinusoids), with 100% sensitivity and 75% specificity predicting a non‐biventricular outcome if three of these four criteria are fulfilled. High‐quality data were available for 20/23 patients: 10 patients had a score of three, four had a score of two, two a score of one and four patients had a score of zero. Significant changes in RV function and dimensions before and after intervention were evaluated using paired t‐tests. Statistical significance was considered as P ≤ 0.05. Fetal cardiac Z‐scores were obtained using a statistical program (Cardio Z; UBQO, Evelina Children's Hospital, London, UK). Data are expressed as median with range, unless stated otherwise.

dimensions before and after intervention were evaluated using paired t‐tests. Statistical significance was considered as P ≤ 0.05. Fetal cardiac Z‐scores were obtained using a statistical program (Cardio Z; UBQO, Evelina Children's Hospital, London, UK). Data are expressed as median with range, unless stated otherwise. RESULTS Between October 2000 and July 2017, a total of 129 fetal intracardiac interventions were performed in 108 patients at our center. Of these, 35 procedures were fetal pulmonary valvuloplasties which were carried out on 23 fetuses with PAIVS (n = 15) or CPS (n = 8). Most interventions (31/35) took place after January 2010. Median gestational age at the time of intervention was 28 + 4 (range, 23 + 6 to 32 + 1) weeks. Patients were referred from nine different countries. Baseline echocardiographic measurements of the patients are presented in Table 1. Table 1 Cardiac measurements, before intervention, of 23 fetuses with pulmonary atresia with intact ventricular septum or critical pulmonary stenosis, that underwent fetal pulmonary valvuloplasty Parameter n * Value (median (range)) RV/LV ratio 19 0.57 (0.47 to 0.70) TV/MV ratio 19 0.69 (0.62 to 0.97) RV filling time† 19 0.28 (0.19 to 0.42) RV length Z‐score 18 –2.95 (–4.64 to –1.33) TV annular diameter Z‐score 18 –1.77 (–3.52 to 0.39) TR Vmax (m/s) 19 4.83 (3.28 to 5.5) TV‐VTI × HR (cm × bpm) 18 821.5 (287.5 to 1859) * Data not available in all cases. † Duration of tricuspid valve (TV) inflow indexed to cardiac cycle length.

Parameter n * Value (median (range)) RV/LV ratio 19 0.57 (0.47 to 0.70) TV/MV ratio 19 0.69 (0.62 to 0.97) RV filling time† 19 0.28 (0.19 to 0.42) RV length Z‐score 18 –2.95 (–4.64 to –1.33) TV annular diameter Z‐score 18 –1.77 (–3.52 to 0.39) TR Vmax (m/s) 19 4.83 (3.28 to 5.5) TV‐VTI × HR (cm × bpm) 18 821.5 (287.5 to 1859) * Data not available in all cases. † Duration of tricuspid valve (TV) inflow indexed to cardiac cycle length. HR, heart rate; RV, right ventricular; RV/LV ratio, right to left ventricular length ratio; TR Vmax, maximum velocity of tricuspid regurgitation; TV/MV ratio, tricuspid to mitral valve annular diameter ratio; TV‐VTI, tricuspid valve velocity time integral. Seventeen patients had a technically successful and four a partially successful procedure (91.3%). The procedure itself was successful in 22/35 (62.9%) interventions. Individual patient data regarding cardiac anatomy and procedural details, complications and outcome are listed in Table 2. Table 2 Procedure details, complications and outcome of fetal pulmonary valvuloplasty (FPV) performed on 23 fetuses with pulmonary atresia with intact ventricular septum (PAIVS) or critical pulmonary stenosis (CPS) Case Cardiac anatomy GA at FPV (weeks) Year of FPV FPV attempts (n) GA at subsequent FPV(s) (weeks) Technical success Balloon/valve ratio Pericardial effusion n.t. Bradycardia n.t.

Table 2 Procedure details, complications and outcome of fetal pulmonary valvuloplasty (FPV) performed on 23 fetuses with pulmonary atresia with intact ventricular septum (PAIVS) or critical pulmonary stenosis (CPS) Case Cardiac anatomy GA at FPV (weeks) Year of FPV FPV attempts (n) GA at subsequent FPV(s) (weeks) Technical success Balloon/valve ratio Pericardial effusion n.t. Bradycardia n.t. Prediction score* before FPV PV gradient (Vmax) after FPV (m/s) GA at birth (weeks) Postnatal procedure(s) Circulation Follow‐up (years) 1† PAIVS 27 + 6 2000 1 — Y 1.0 N Y 3 ND 37 + 6 PV dilatation, modified BT shunt, BT shunt removal at 9 months BV 16.36 2 CPS 24 + 0 2003 1 — N — N N ND — 32 + 0 Modified BT shunt UV NND 3† PAIVS 32 + 1 2007 2 31 + 2 Y ND N N ND ND 37 + 1 PV dilatation, modified BT shunt, BDG and RVOT enlargement 1.5 V 9.28 4 CPS 28 + 5 2010 1 — Y 1.0 N N 3 3.1 40 + 3 PV dilatation, RVOT patch, PV valvotomy and modified BT shunt, coiling BT shunt BV 7.16 5 CPS 29 + 3 2010 1 — Y 1.0 N N 0 4.4 38 + 3 PV dilatation BV 6.82 6† PAIVS 25 + 0 2012 1 — Y 1.1 N N 3 1.3 37 + 1 Valvotomy and modified BT shunt, coiling BT shunt BV 4.78 7 PAIVS 31 + 0 2013 1 — N — N N 3 — ND Modified BT shunt followed by BDG anastomosis and Fontan procedure UV LFU 8 CPS 26 + 6 2013 2 25 + 4 Partial 1.0 N N 3 2.1 35 + 1 Modified BT shunt, PV dilatation, BDG and RPA patch 1.5 V 3.56 9 PAIVS 25 + 5 2014 1 — Y 0.9 N N 3 3.5 40 + 4 PV dilatation (4×) BV 2.40 10 PAIVS 28 + 5 2014 1 — Y 1.1 Y Y 2 2.1 37 + 5 PV dilatation BV 2.75 11† CPS 29 + 3 2015 1 — Y 0.9 Y N 0 2.2 39 + 1 PV dilatation BV 1.81 12 PAIVS 27 + 5 2015 2 27 + 0 Y 0.8 N Y 3 3.3 39 + 1 Radiofrequency perforation PV, modified BT shunt, PV dilatation Undetermined 1.61 13† PAIVS 23 + 6 2015 3 23 + 5, 28 + 5 Partial 0.6 N Y 3 1.2 38 + 2 PV perforation and dilatation, modified BT shunt and RVPAC, BDG and RVPAC 1.5 V 1.63 14 CPS 25 + 1 2015 2 26 + 2 Partial 0.9 N Y 0 3.4 38 + 3 PV dilatation BV 1.64 15 PAIVS 30 + 4 2016 2 30 + 1 Y 0.7 Y Y 2 2.9 38 + 1 PV dilatation (2×) BV 1.25 16 PAIVS 28 + 3 2016 2 26 + 4 Y 1.1 Y Y 2 ND 38 + 4 PV perforation and dilatation, PDA stent, PV dilatation (2×), spontaneous closure of PDA stent, PV dilatation BV 1.13 17 PAIVS 27 + 3 2016 1 — Y 1.2 N N 2 3.7 38 + 0 PV dilatation (2×) BV 0.90 18† CPS 30 + 1 2016 1 — Y 0.8 N N 1 1.3 33 + 5 PV dilatation (2×) BV 0.93 19† PAIVS 27 + 4 2016 3 22 + 3, 26 + 4 Partial 0.8 N N 3 4.0 39 + 3 PV perforation and dilatation, PDA stent and PV dilatation BV 0.78 20 PAIVS 28 + 4 2016 2 28 + 3 Y ND N Y ND ND 39 + 1 Modified BT shunt and RVOT enlargement Undetermined

BV 0.90 18† CPS 30 + 1 2016 1 — Y 0.8 N N 1 1.3 33 + 5 PV dilatation (2×) BV 0.93 19† PAIVS 27 + 4 2016 3 22 + 3, 26 + 4 Partial 0.8 N N 3 4.0 39 + 3 PV perforation and dilatation, PDA stent and PV dilatation BV 0.78 20 PAIVS 28 + 4 2016 2 28 + 3 Y ND N Y ND ND 39 + 1 Modified BT shunt and RVOT enlargement Undetermined 0.22 21† PAIVS 29 + 1 2017 1 — Y 1.1 N N 1 2.8 30 + 4 No intervention BV 0.1 NND 22 CPS 28 + 3 2017 1 — Y ND N Y 0 1.6 35 + 5 PV dilatation BV 0.24 23 PAIVS 29 + 5 2017 2 29 + 0 Y 1.0 N N 3 3.5 30 + 1 On i.v. prostaglandins at last follow‐up Undetermined 0.11 * Assessed by four‐point scoring system of Roman et al.15 (criteria: tricuspid/mitral valve annular diameter ratio < 0.7, RV/LV length ratio < 0.6, RV filling time < 31.5% of cardiac cycle length, presence of RV sinusoids) indicating non‐BV circulation after birth for fetuses with score of 3 with 100% sensitivity and 75% specificity. † Patient followed up and treated at our institution. 1.5 V, one‐and‐a‐half ventricle circulation; BDG, bidirectional Glenn anastomosis; BT shunt, Blalock–Taussig shunt; BV, biventricular; GA, gestational age; i.v., intravenous; LFU, lost to follow‐up; LV, left ventricle; N, no; ND, no data available; NND, neonatal death; n.t., necessitating treatment; PDA, persistent ductus arteriosus; PV, pulmonary valve; RPA, right pulmonary artery; RV, right ventricle; RVOT, right ventricular outflow tract; RVPAC, right ventricle‐to‐pulmonary artery conduit; TV, tricuspid valve; UV, univentricular; Y, yes.

ND, no data available; NND, neonatal death; n.t., necessitating treatment; PDA, persistent ductus arteriosus; PV, pulmonary valve; RPA, right pulmonary artery; RV, right ventricle; RVOT, right ventricular outflow tract; RVPAC, right ventricle‐to‐pulmonary artery conduit; TV, tricuspid valve; UV, univentricular; Y, yes. Mortality, complications and prematurity No fetal death occurred. All 23 fetuses were liveborn at a median gestational age of 38 + 2 (range, 30 + 1 to 40 + 4) weeks. Pericardial effusion necessitating treatment occurred in 4/35 (11.4%) and persistent bradycardia in 11/35 (31.4%) procedures. There were six (26.1%) preterm deliveries, two due to premature rupture of membranes (3 days after intervention at 30 + 2 weeks and 8 weeks after intervention at 33 weeks), one after 3 weeks of intervention due to pre‐eclampsia of the mother and three due to preterm labor between 9 days and 8 + 2 weeks after procedure. Additionally, one neonate was found to have CHARGE syndrome with esophageal atresia.

(3 days after intervention at 30 + 2 weeks and 8 weeks after intervention at 33 weeks), one after 3 weeks of intervention due to pre‐eclampsia of the mother and three due to preterm labor between 9 days and 8 + 2 weeks after procedure. Additionally, one neonate was found to have CHARGE syndrome with esophageal atresia. Immediate changes after fetal pulmonary valvuloplasty For the 17 fetuses that had successful pulmonary valvuloplasty, all cardiac parameters improved significantly after the intervention (Figure 1); the most significant change was observed for RV filling time. Cardiac measurements before and after successful intervention are shown in Table 3. Figure 2 shows change in RV dimensions and improvement of RV filling time with change from a short monophasic to a longer biphasic inflow and Figures 3a and b illustrate decrease of TR velocity after intervention. Even after a successful intervention, there remained a gradient across the PV in almost all cases (Table 2). Figure 3c shows the residual PV gradient with a new pulmonary regurgitation immediately after a successful procedure. Fetuses with a partially successful intervention (n = 4) demonstrated improvement in cardiac parameters as well, but not in all parameters (Table S1).

adient across the PV in almost all cases (Table 2). Figure 3c shows the residual PV gradient with a new pulmonary regurgitation immediately after a successful procedure. Fetuses with a partially successful intervention (n = 4) demonstrated improvement in cardiac parameters as well, but not in all parameters (Table S1). Figure 1 Cardiac measurements before and after completely or partially successful fetal pulmonary valve intervention in 21 fetuses with pulmonary atresia with intact ventricular septum or critical pulmonary stenosis. Patients 6 and 23 were excluded from (c) and (e) due to exclusive retrograde filling of right ventricle by severe pulmonary regurgitation. HR, heart rate; RV/LV ratio, right to left ventricular length ratio; TR Vmax, maximum velocity of tricuspid regurgitation; TV/MV ratio, tricuspid to mitral valve annular diameter ratio; TV‐VTI, tricuspid valve velocity time integral. UOG-19047-FIG-0001-bTable 3 Cardiac measurements before and after completely successful intervention in 17 fetuses that underwent fetal pulmonary valvuloplasty for pulmonary atresia with intact ventricular septum or critical pulmonary stenosis Parameter n * Value (median (range)) P

Figure 1 Cardiac measurements before and after completely or partially successful fetal pulmonary valve intervention in 21 fetuses with pulmonary atresia with intact ventricular septum or critical pulmonary stenosis. Patients 6 and 23 were excluded from (c) and (e) due to exclusive retrograde filling of right ventricle by severe pulmonary regurgitation. HR, heart rate; RV/LV ratio, right to left ventricular length ratio; TR Vmax, maximum velocity of tricuspid regurgitation; TV/MV ratio, tricuspid to mitral valve annular diameter ratio; TV‐VTI, tricuspid valve velocity time integral. UOG-19047-FIG-0001-bTable 3 Cardiac measurements before and after completely successful intervention in 17 fetuses that underwent fetal pulmonary valvuloplasty for pulmonary atresia with intact ventricular septum or critical pulmonary stenosis Parameter n * Value (median (range)) P Before intervention After intervention RV/LV ratio 14 0.57 (0.47 to 0.70) 0.64 (0.58 to 0.75)  0.0001 TV/MV ratio 13 0.69 (0.62 to 0.83) 0.78 (0.68 to 0.88)  0.0004 RV filling time† 13 0.28 (0.19 to 0.42) 0.38 (0.33 to 0.54)  0.000002 RV length Z‐score 13 –2.95 (–4.64 to –1.33) –2.08 (–3.85 to –1.09)  0.0005 TV annular diameter Z‐score 13 –1.81 (–3.52 to –1.31) –1.56 (–2.55 to –0.53)  0.0012 TR Vmax 12 4.88 (3.28 to 5.50) 3.90 (2.74 to 5.38)  0.0008 TV‐VTI × HR (cm × bpm) 11 868 (287.5 to 1859) 1044 (650 to 1687.5)  0.008 * Data not available in all cases. † Duration of tricuspid valve (TV) inflow indexed to cardiac cycle length.

Before intervention After intervention RV/LV ratio 14 0.57 (0.47 to 0.70) 0.64 (0.58 to 0.75)  0.0001 TV/MV ratio 13 0.69 (0.62 to 0.83) 0.78 (0.68 to 0.88)  0.0004 RV filling time† 13 0.28 (0.19 to 0.42) 0.38 (0.33 to 0.54)  0.000002 RV length Z‐score 13 –2.95 (–4.64 to –1.33) –2.08 (–3.85 to –1.09)  0.0005 TV annular diameter Z‐score 13 –1.81 (–3.52 to –1.31) –1.56 (–2.55 to –0.53)  0.0012 TR Vmax 12 4.88 (3.28 to 5.50) 3.90 (2.74 to 5.38)  0.0008 TV‐VTI × HR (cm × bpm) 11 868 (287.5 to 1859) 1044 (650 to 1687.5)  0.008 * Data not available in all cases. † Duration of tricuspid valve (TV) inflow indexed to cardiac cycle length. HR, heart rate; RV, right ventricular; RV/LV ratio, right to left ventricular length ratio; TR Vmax, maximum velocity of tricuspid regurgitation; TV/MV ratio, tricuspid to mitral valve annular diameter ratio; TV‐VTI, tricuspid valve velocity time integral. Figure 2 Apical four‐chamber views of Patient 9 before (a) and 2 days after (b) intervention; arrows mark points of measurement for tricuspid and mitral valve annular diameter and right (RV) and left (LV) ventricular length; note increase in RV length and tricuspid valve diameter after intervention. Pulsed‐wave Doppler traces with RV filling times of Patient 15 before (c) and after (d) intervention; note change from short monophasic RV filling of 25% (107/422 ms) of cardiac cycle length before intervention to longer biphasic RV filling of 39% (197/505 ms) of cardiac cycle length after intervention. LA, left atrium; RA, right atrium; TR, tricuspid regurgitation.

of Patient 15 before (c) and after (d) intervention; note change from short monophasic RV filling of 25% (107/422 ms) of cardiac cycle length before intervention to longer biphasic RV filling of 39% (197/505 ms) of cardiac cycle length after intervention. LA, left atrium; RA, right atrium; TR, tricuspid regurgitation. UOG-19047-FIG-0002-cFigure 3 Continuous‐wave (CW) Doppler traces of tricuspid regurgitation (TR) in Patient 11 before and after intervention; pressure gradient was reduced from 75 mmHg (4.34 m/s) (a) to 49 mmHg (3.50 m/s) (b). (c) CW Doppler tracing of blood flow across pulmonary valve (PV) with remaining antegrade gradient of 49 mmHg and new pulmonary regurgitation (PR) in Patient 9 after intervention (c). UOG-19047-FIG-0003-cIntrauterine course Longitudinal data of cases with successful or partially successful intervention were available from five patients who were all delivered and followed up in our institution. Preintervention, immediate postintervention and first postpartum measurements are shown in Figure 4. The greatest change in RV/LV ratio occurred immediately after the procedure and it continued to increase in three patients, whereas it remained stable in the other two patients. TV/MV ratio after intervention continued to increase in four patients.

stintervention and first postpartum measurements are shown in Figure 4. The greatest change in RV/LV ratio occurred immediately after the procedure and it continued to increase in three patients, whereas it remained stable in the other two patients. TV/MV ratio after intervention continued to increase in four patients. Figure 4 Longitudinal development of right to left ventricular length (RV/LV) ratio (a) and tricuspid to mitral valve annular diameter (TV/MV) ratio (b) in five fetuses that underwent fetal pulmonary valvuloplasty. Plotted are measurements before intervention, immediately after intervention and immediately postpartum, with intervention indicated by asterisks. In Patients 11 and 19, two measurements were performed before intervention. Measurement of TV/MV ratio immediately after intervention is missing for Patient 19 (b). , Patient 6; , Patient 11; , Patient 18; , Patient 19; , Patient 21. UOG-19047-FIG-0004-bProgressive PV stenosis during advancing gestation was observed in almost all fetuses and re‐atresia occurred in four patients. Two of those fetuses had a successful intervention at 27 + 5 and 28 + 3 weeks with a balloon‐to‐valve ratio of 0.8 and 1.1, respectively. The remaining two had partially successful interventions at 23 + 6 and 27 + 4 weeks with balloon‐to‐valve ratios of 0.6 and 0.8, respectively.

d re‐atresia occurred in four patients. Two of those fetuses had a successful intervention at 27 + 5 and 28 + 3 weeks with a balloon‐to‐valve ratio of 0.8 and 1.1, respectively. The remaining two had partially successful interventions at 23 + 6 and 27 + 4 weeks with balloon‐to‐valve ratios of 0.6 and 0.8, respectively. Postnatal outcome Postnatal procedures and outcome at last follow‐up for all patients are listed in Table 2. There were two neonatal deaths, both of which occurred after a preterm delivery. Patient 2 was delivered at 32 weeks following an unsuccessful intervention, and was diagnosed with CHARGE syndrome and died after placement of a BT shunt due to hemodynamic instability. Patient 21 was delivered at 30 + 4 weeks, 9 days after a successful intervention, and was found to have a good sized RV with wide open PV; the arterial duct remained open without prostaglandins and the patient rapidly developed severe necrotizing enterocolitis and died after extensive bowel resection.

amic instability. Patient 21 was delivered at 30 + 4 weeks, 9 days after a successful intervention, and was found to have a good sized RV with wide open PV; the arterial duct remained open without prostaglandins and the patient rapidly developed severe necrotizing enterocolitis and died after extensive bowel resection. All other patients were alive after a median follow‐up of 1.63 (range, 0.10–16.36) years. Freedom from mortality after 1 year was 90.2%. Out of the 21 fetuses with a successful or partially successful procedure, with various prediction scores, 15 (71.4%) became biventricular, three (14.3%) had a one‐and‐a‐half ventricle circulation and three (14.3%) had an undetermined circulation. One infant was still on prostaglandins 6 weeks after a preterm birth at 31 weeks and two patients had a modified BT shunt at the age of 2.6 and 19.3 months. All but one of the neonates needed an early PV intervention and 11 additionally required a systemic‐to‐pulmonary‐artery shunt or a ductal stent in the neonatal period. Fetuses with predicted non‐biventricular outcome The prediction score as described by Roman et al.15 could be calculated for 20 of 23 fetuses (Figure 5). Of these, 10 (50%) had a score of three, which is considered to predict a non‐biventricular circulation after birth with 100% sensitivity. Six of these 10 patients had a successful intervention, four of which achieved a biventricular circulation and two had an undetermined circulation at the age of 41 days and 1.6 years.

these, 10 (50%) had a score of three, which is considered to predict a non‐biventricular circulation after birth with 100% sensitivity. Six of these 10 patients had a successful intervention, four of which achieved a biventricular circulation and two had an undetermined circulation at the age of 41 days and 1.6 years. Figure 5 Circulation outcome in 23 fetuses that underwent fetal pulmonary valvuloplasty according to prediction score by Roman et al.15, a four‐point scoring system indicating non‐biventricular (BV) circulation after birth for fetuses with score of 3, with 100% sensitivity and 75% specificity. Note that 5/10 patients with predicted non‐BV outcome had BV circulation. 1.5 V, one‐and‐a‐half ventricle circulation; UV, univentricular circulation. UOG-19047-FIG-0005-bDISCUSSION In the absence of major coronary fistulae or muscular atresia of the RV outflow tract, RV and TV size at birth are the major determinants for a biventricular outcome. We have shown that technically successful pulmonary valvuloplasty in second‐ and third‐trimester fetuses with PAIVS or CPS leads to an immediate and significant increase in RV and TV size, decrease in RV pressure and longer and better RV filling. In the five fetuses that were followed up in our institution, we observed that initial cardiac changes were followed by continued growth of RV structures until birth. In more than half of the fetuses with a predicted non‐biventricular outcome and successful procedure, a two‐ventricle circulation was achieved.

filling. In the five fetuses that were followed up in our institution, we observed that initial cardiac changes were followed by continued growth of RV structures until birth. In more than half of the fetuses with a predicted non‐biventricular outcome and successful procedure, a two‐ventricle circulation was achieved. In all fetuses with a successful PV intervention, significantly greater RV length and TV diameter were observed 1 to 2 days after the procedure, which has not been described before. We do not believe that real RV or TV growth had occurred in such a short period of time, but we suggest that the decompression of the RV led immediately to improved filling, evidenced by the significantly increased TV‐VTI × HR product, resulting in greater RV volumes. Obviously, RV length had been underestimated before the intervention, because the RV apex was formed by the moderator band. Increased filling after intervention revealed some cavity at the RV apex, which was not visible before. The increase of TV diameter can be explained by better and longer TV opening, which made it easier to measure the full diameter of the valve. This is an important finding, because underestimation of the true RV and TV size may lead to prediction of a poorer outcome resulting in overly pessimistic counseling of the parents. Reduction in RV pressure was also highly significant, as estimated by TR velocity and the lengthening of RV filling time, both logical consequences of establishing and improving forward flow through the PV. Despite being lower than preintervention, RV pressure remained above normal in most cases. We were not able to eliminate completely the gradient across the pulmonary outflow tract, particularly in late gestation, mainly because the available balloons were too small due to the limited inner diameter of the needles we used. In fact, we think that a balloon‐to‐valve ratio of 1.3–1.5 is necessary to perform an optimal pulmonary valvuloplasty. Even when we considered the intervention as only partially successful, immediate improvements could be observed, indicating that even a minor RV decompression is able to improve hemodynamics.

we used. In fact, we think that a balloon‐to‐valve ratio of 1.3–1.5 is necessary to perform an optimal pulmonary valvuloplasty. Even when we considered the intervention as only partially successful, immediate improvements could be observed, indicating that even a minor RV decompression is able to improve hemodynamics. No procedure‐related deaths or intrauterine deaths occurred at our center. Certainly, right heart interventions are better tolerated than are left heart ones, and a key advantage was that all procedures were carried out by an experienced team that had already performed a substantial number of left heart interventions16, 17.

ocedure‐related deaths or intrauterine deaths occurred at our center. Certainly, right heart interventions are better tolerated than are left heart ones, and a key advantage was that all procedures were carried out by an experienced team that had already performed a substantial number of left heart interventions16, 17. An important question of this study was whether the initial increase in RV size would be followed by natural growth of right heart structures. It has been shown in animal experiments that the fetal myocardium responds with myocyte proliferation, i.e. hyperplasia, to increased pre‐ and afterload18, 19, 20. The ability for myocyte proliferation is lost shortly after birth and the postnatal myocardium responds with hypertrophy rather than hyperplasia. After neonatal pulmonary valvuloplasty, the RV will grow but only in relation to body size and catch‐up growth does not occur7. Our limited longitudinal data provide evidence that, after a period of little or no RV growth, a successful PV intervention could have the potential to restore RV growth towards term, a finding that is in agreement with observations in previous small studies8, 10, 11. Three out of five fetuses in our cohort even demonstrated a further increase in RV/LV ratio and four of five patients a further increase in TV/MV ratio, indicating that there could even be an additional catch‐up growth of the RV prenatally. We speculate that this RV growth is achieved by myocardial hyperplasia, which would not have been possible after birth. Theoretically, at birth, these RVs should consist of more myocardial cells with a potential for better postnatal adaptation and improved long‐term function.

ional catch‐up growth of the RV prenatally. We speculate that this RV growth is achieved by myocardial hyperplasia, which would not have been possible after birth. Theoretically, at birth, these RVs should consist of more myocardial cells with a potential for better postnatal adaptation and improved long‐term function. During advancing gestation, progressive gradients across the PV were observed in almost all fetuses. This could have been due to real re‐stenosis of the dilated PV or related to progressive better RV filling and thus increased flow across the valve (relative stenosis). Re‐atresia of the PV was observed in four fetuses. Two fetuses had only partially successful interventions early in gestation, so the created opening of the PV was obviously too small to stay open until birth. Interestingly, the two other fetuses initially had successful interventions and also developed atresia. It is possible that the balloon‐to‐valve ratios of 0.8 and 1.1 were still too small and this could have been avoided with larger balloons.

e created opening of the PV was obviously too small to stay open until birth. Interestingly, the two other fetuses initially had successful interventions and also developed atresia. It is possible that the balloon‐to‐valve ratios of 0.8 and 1.1 were still too small and this could have been avoided with larger balloons. In the absence of a valid control group during the same study period, we chose to refer to published criteria for prediction of outcome. Several reports have been published regarding the prediction of a non‐biventricular/univentricular outcome in fetuses with PAIVS/CPS5, 11, 15, 21. Using the criteria published by Roman et al.15, we found that more than half of the patients with a predicted non‐biventricular outcome could achieve a biventricular circulation. This provides evidence that a timely in‐utero pulmonary valvuloplasty is able to modify the natural history of fetuses with PAIVS or CPS and increases the likelihood of biventricular circulation after birth. However, as normal RV size could not be achieved even after a successful procedure, it is justified to avoid delaying intervention until the RV becomes severely hypoplastic, and instead perform fetal pulmonary valvuloplasty as soon as possible to allow optimal in‐utero recovery.

f biventricular circulation after birth. However, as normal RV size could not be achieved even after a successful procedure, it is justified to avoid delaying intervention until the RV becomes severely hypoplastic, and instead perform fetal pulmonary valvuloplasty as soon as possible to allow optimal in‐utero recovery. The most important limitations of this study are its retrospective design, the small number of patients and the non‐standardized postnatal management of the respective referral centers. A further limitation is the lack of a control group of matched fetuses that did not undergo fetal pulmonary valvuloplasty; data published by Roman et al.15 were used instead. Furthermore, reliable longitudinal data were available for only five out of 23 patients; therefore, no significance of continuous RV growth could be calculated, and certainly a greater number of patients would be needed to confirm these observations. In conclusion, for selected fetuses with PAIVS or CPS, in‐utero pulmonary valvuloplasty leads immediately to larger RV caused by reduced afterload and increased filling, thus improving the likelihood of a biventricular outcome. Supporting information Table S1 Cardiac measurements of four fetuses before and after partially successful fetal pulmonary valvuloplasty Click here for additional data file.

INTRODUCTION The term lower urinary tract obstruction (LUTO) refers to a heterogeneous group of anatomical anomalies causing an obstruction in the urethra1. During fetal life, LUTO entails a sequence of events that are detectable on antenatal ultrasound examination. This typically starts with evidence of a distended bladder (megacystis) accompanied by hydronephrosis, progressing to renal dysplasia with abnormal renal parenchymal appearance on ultrasound examination and eventually resulting in severe oligohydramnios2. The condition is associated with a high rate of mortality and postnatal morbidity due to lung hypoplasia and impaired renal function3. When LUTO is suspected in the first trimester and megacystis >12 mm is seen, the prognosis is extremely poor and parents often opt for termination of pregnancy4, 5. For cases identified later in pregnancy, no definitive criteria for diagnosing LUTO and predicting the precise prognosis have yet been proposed6. Beyond the first trimester, the diagnosis of LUTO is typically based on the evidence of three ultrasound findings: megacystis, dilated posterior urethra (known as the keyhole sign), and either unilateral or bilateral hydronephrosis.

teria for diagnosing LUTO and predicting the precise prognosis have yet been proposed6. Beyond the first trimester, the diagnosis of LUTO is typically based on the evidence of three ultrasound findings: megacystis, dilated posterior urethra (known as the keyhole sign), and either unilateral or bilateral hydronephrosis. Over the past 20 years, fetal therapy has been attempted based on the assumption that, by relieving the intracavitary pressure caused by the obstruction, mortality and renal damage could possibly be prevented. The PLUTO trial investigated this assumption, demonstrating a significant improvement in survival of fetuses treated with vesicoamniotic shunt, but reporting a high rate of morbidity among survivors, irrespective of the antenatal management7. To date, whether and when in‐utero treatment should be offered remains a matter of debate, and the eventual selection of candidates is still suboptimal, owing to the high number of false‐positive LUTO cases8. In fact, a previous study reported that one‐third of all LUTO cases suspected prenatally are reclassified postnatally, primarily to vesicoureteral reflux9. For this reason, an improvement in the diagnostic accuracy of ultrasound for LUTO is called for. The aim of this study was to identify the optimal combination of ultrasound parameters for the antenatal diagnosis of LUTO from the second trimester, as an alternative to the commonly used LUTO triad (megacystis, keyhole sign and hydronephrosis).

ic transformations were used where appropriate. t‐tests or the Mann–Whitney U‐test, and Fisher's exact test were used to test differences between groups. Statistical analysis was performed using statistical package Stata Version 13 (StataCorp., College Station, TX, USA) and SPSS Version 21 (IBM Corp., Armonk, NY, USA). RESULTS Three hundred and ninety‐six women were enrolled in the study; plasma samples from 396 and serum samples from 244 women were assayed. Baseline characteristics at booking and study enrolment are presented in Table 1 and maternal outcomes are shown in Table 2. Overall, 62 (16%) women had a final diagnosis of preterm pre‐eclampsia. DELFIA Xpress PlGF 1‐2‐3 test concentrations were examined and a concentration of < 150 pg/mL was determined to give an optimal test performance, with the same overall proportion of positive tests (37%) as the triage PlGF test, without regard to the final diagnosis. This value (150 pg/mL) was used for comparisons between tests. Table 1 Maternal characteristics, at booking and enrolment, of 396 women with singleton pregnancy and suspected preterm pre‐eclampsia, according to gestational age at time of presentation