Literature Review on Obstetric Ultrasound Diagnostic Accuracy

The role of quality assurance and quality control in improving the diagnostic accuracy of obstetric ultrasound: A Literature review.

Abstract

The importance of achieving accurate foetal measurements in obstetric ultrasound is widely understood to allow high quality assessment of foetal growth and development. Ultrasound has been widely accepted as the routine method of diagnosis in the obstetric field for foetal abnormalities due to its accessibility and efficiency in diagnosis. Yet, the operation dependent nature of ultrasound causes calculations to be prone to error that may drastically affect foetal measurements recorded and the outcome of the foetus. In this review, various studies focused on quality assurance and quality control are summarised with their results and limitations discussed. Furthermore, related issues including the potential of bias in self appraisal of image quality, inaccurate growth charts and operator related factors are highlighted to be hindering the progress of the optimisation of obstetric ultrasound.  Additionally, future research in nuchal translucency and foetal biometry acquisition regarding the vast developments in ultrasound technology and techniques proposed to increase ease of use and diagnostic accuracy will be discussed. Literature was obtained from online databases such as Ovid Medline, Scopus and PubMed using keywords such as "Obstetric ultrasound", "Quality assurance" and "Quality control". The scope of literature was broadened to compensate by discussing quality assurance measurements at different gestational phases; such as "Fetal biometry" and "Nuchal Translucency", as well as including experimental designs focused on quality assurance of image reporting, as optimisation of images are not limited to the image collection but also the interpretation and reporting of those images. In a review of current literature addressing the quality control and assurance of obstetric sonography, a common agreement is that implementation of quality assurance measures will increase fetal measurement reliability significantly. Although there is variability to the strategies, there is still a lack of a standardised framework for the assessment of quality assurance across clinical sites of the modality.

Introduction

Obstetric sonography is a major modality used for the measurement of fetal weight, anatomical anomalies, and general fetal wellbeing. This modality has seen an exponential growth in application and technological advancements however, optimisation and guarantee of image quality remains an overlooked and neglected aspect in literature due to poor resources and efficiency associated with quality assurance measures. However, there is an importance for image quality measures to address the variance of the quality of studies and consequences due to the operator dependent nature of sonography. Assessment and guarantee of image quality involve the implementation of quality assurance and quality control systems (Salomon). This literature review intends to explore the current state of quality assurance measures in obstetric sonography, describe standardised improvements to inter and intra-operator reliability, and identify why quality assurance and quality control systems are so important for obstetric ultrasound. Quality assurance will be defined as the implementation of systematic and planned tasks in a system to demonstrate a determined standard of image quality and their conditions will be met consistently (Salomon). This system aims to guarantee consistent acquisition of high-quality images. Quality control is defined as evaluations of techniques or activities that monitor quality to ensure consistent high-quality delivery of images (Hediger M). Resource management, such as cost efficiency, time constraint and effective equipment usage are highly prioritised in the increasingly demanding setting of the medical field (Salomon). Though the sensitivity and precision of fetal anatomy are higher in other modalities such as Magnetic Resonance Imaging (MRI), sonography is the primary modality fetal measurement for its faster image acquisition and low cost. This prioritisation raises questions in various literature on the practicality and impact of implementing image quality measures, despite some literature suggestion of increased reliability and accuracy of measurements from the application of various quality assurance and control methods (Russell S).

Process of analysing literature

Literature was obtained from online databases such as Ovid Medline, Scopus and PubMed using keywords such as "Obstetric ultrasound", "Quality assurance" and "Quality control" for a broad topic search. However, the literature was limited, as the topic is largely neglected. Therefore the scope of literature was broadened to compensate by discussing quality assurance measurements at different gestational phases; such as "Fetal biometry" and "Nuchal Translucency", as well as including experimental designs focused on quality assurance of image reporting, as optimisation of images are not limited to the image collection but also the interpretation and reporting of those images (1). Sources were selected based on peer-review, date of publication, experimental design and whether it was a large-scale study to ensure that the ultrasound protocol reviewed is the most recent and applicable to modern settings. Measurements of nuchal translucency (NT) and crown rump length (CRL) were chosen as the preferred parameters for obstetric sonography of the first trimester to calculate the gestational age, and foetal biometry measurements for the later trimesters to characterise foetal growth rate.

Quality Assurance and Control

Sonography is a preferred modality for routine prenatal measurement and diagnosis for its properties of limited biological harm with moderate use and cost efficiency, however, it is lacking in peer-review and instant evidence of a competent scan or measurement (3). Maintenance and monitoring of measurements in obstetric sonography is crucial, as the impact of biometric measurement errors can lead to misdiagnosis and possible complications for the fetus (Wright)(Warska). A study by Gardosi et al. assessing the maternal and fetal risk factor for stillbirth in a population study, emphasises the importance of accurate fetal biometric measurements, linking missed small gestational aged fetuses to a seven-fold higher risk factor for stillbirths.

Rather than using a strict routine protocol, it is observed amongst healthcare facilities of a lacklustre approach in adopting local or physics-based quality assurance (3). Beyond this, there are limited quality assurance procedures available, and the operators responsible for the implementation of these procedures tend to be inexperienced and overlook faults in the system (3). Benacerraf et al. ⁽⁴⁾ suggested dedicating time for quality assurance and case reviews will promote maintaining and monitoring of operator skills on an ongoing basis, and promotes proper operator certification and site accreditation to inform and familiarise operators with standard quality assurance and control procedures. Conversely, Benacerraf et al. also notes the introduction of new performance requirements imposed on normal clinical workflow would contribute to excess workload for ultrasound operators ^(4).

First Trimester Measurements

The gestational age is ideally found during the first trimester due to the reduced biologic variation in foetal size compared with later trimesters ^(11). This is achieved through Crown Length and Nuchal Translucency measurements, both measurements are compared to published reference charts which correlate to a specific growth period. Both measurements are highly dependent on calliper placement and operator. Studies reviewed focusing on the influence of quality assurance and quality control across foetal ultrasound areas including nuchal translucency and foetal biometry have been implemented and suggest possible benefits. Crown rump length and NT work together as normal thickness of the subcutaneous space depends on the crown rump length of the foetus.

Nuchal Translucency (NT)

Nuchal Translucency measures the subcutaneous space behind the neck of the foetus and the overlying skin (6), assessing the fluid that fills this region. High levels of this fluid are attributed to chromosomal abnormalities and foetal malformations which have a poor prognosis (5), such as Trisomy 21 and Turners syndrome. In optimal conditions, it is demonstrated that at 11-13 weeks, NT testing can diagnose more than 95% of major chromosomal abnormalities and malformations, to allow for an earlier and safer form of possible termination and higher incidence and detection of abnormal accumulation of nuchal fluid  ⁽⁵⁾ . Therefore, it is extremely important for the foetus’ prognosis that proper measures are taken to ensure these ideal conditions are met to allow for an accurate diagnosis.

Crown Rump Length (CRL)

Crown Rump Length involves the measurement of the length of the fetus head, which is used to determine the stage and normality of the growth, with the ideal measurements for the minimum and maximum foetal crown-rump length are 45 mm and 84 mm respectively ⁽⁵⁾. CRL is measured at 8–14 weeks for the estimation of gestational age ⁽¹⁰⁾, as the use of the last menstrual cycle is not an accurate method. This measurement is then compared to values in published reference charts which correlate to different stages of growth. Accuracy in CRL measurements is essential, as a one- or two-day variation greatly alters the calculated risk (12) of the foetus, which has implications for pregnancies at risk of premature delivery or growth restrictions (13). This stresses the importance of a QA system as studies (13) have observed a systematic overestimation of an average three days primarily from assessment of In Vitro Pregnancies (IVF) when using local CRL reference charts. Another issue with the use of local CRL reference charts is the variability between these charts, as they can have significant disagreements and the predicted gestational age varies significantly by several days depending on the chart used (11), therefore CRL measurement accuracy is also dependent on the chart used for comparison.

Quality Assurance in First Trimester Measurements (FTM)

As discussed, the ability to achieve accurate FTM measurements is highly dependent on proper training and adherence to a standard technique to achieve consistency of results among different operators. To guarantee image quality and accurate measurements, a complete audit is usually used across healthcare sites, which involves assessing the distribution of measurements for CRL and NT, in addition to an analysis of image quality of random images from individual sonographers. There are three main factors which are considered when analysing an individual sonographer’s accuracy; median bias, trend and spread, of which each describe a numeric value (14). Median bias refers to the proportion of spread of NT data points with respect to the vertical axis and relative to the standardised median curve. A satisfactory audit will have 40-60% of nuchal translucency measurements above the median, and images graded as high quality based on qualitative criteria ^{(1) (15)}.  The spread of scores describes how closely the NT data points enfold the median curve. For a normal population, the NT scores should cluster around the median curve with some fluctuation ⁽¹⁴⁾. Tight and wide degree spread can indicate bias in choosing CRL to match NT values and inconsistent NT values overall respectively. Trend is used to describe the shape of the NT data distribution compared to the median curve ⁽¹⁴⁾. In an ideal scenario, the NT distribution should mirror the shape and direction of the median curve. A steep positive trend refers to when most NT data points are lower than the median curve at smaller CRLs (undermeasurement) and higher at larger CRLs (overmeasurement) ^(14).  A steep negative trend indicates most NT data points are higher than median curve at smaller CRLs (overmeasurement) and lower than the median curve at larger CRLs (undermeasurement). Flattened trend refers to most NT data points to present on the same line with respect to the horizontal axis. A flattened trend implies that the operator is consistently obtaining the same NT measurement across all CRLs with limited variation ^(14). Literature reveal that NT has common issue with negative bias affecting recorded measurements ⁽¹⁶⁾. The Better Outcomes Registry and Network Ontario states a multitude of reasons to result in negative bias ranging from; incorrect calliper placement, failing to measure the widest portion of the NT,  not recording the largest NT measurement, over gaining of the image causing fill-in of the anechoic NT, image acquisition that was not taken in midline sagittal plane of the foetus and the inadequate use of zoom ^(14). Studies have observed that grouped assessment of NT images from an individual sonographer rather than examining individual NT images is time efficient and allows more personalised feedback, especially for suboptimal images ⁽¹⁷⁾.

Ultrasound examination of the foetus is a subjective process that is very reliant on operator skills and the quality of the sonographic equipment available ^(6). The ability to achieve accurate NT measurements is dependent on proper training and adherence to a standard technique to achieve consistency of results among different operators ^{(5) (18)}. Methods amongst studies to achieve standardisation of measurements include education programs for sonographers and radiologists and the updating and vigilance of operator medians for quality assurance. Nisbet et al. ⁽¹⁵⁾ recommended the use of online resources such as teleconference tutorials as an easily accessible and effective way to promote instant and continuous improvement in operator performance.  Other studies ⁽¹⁸⁾ support that constant quality assurance is needed, instituting sonographer-specific medians and providing individualised feedback about performance. Several operator and foetal parameters have a significant impact on the quality standards ranging from operator experience and certification and the range of CRL measurements used.  Axell et al. ⁽¹²⁾ investigated whether the precision of ultrasound measurements can be affected by machine–probe combinations and their study results suggested that they led to greater variability than those ascribed to intraobserver differences. Yet, these findings need further investigation as the study design did not prevent sources of operator bias ⁽¹²⁾. Although even with strict training programs for all sonographers, NT measurements are demonstrated to drift over time for known and unknown reasons ^(18).

Second and Third Trimester Measurements

Second and third trimester obstetric ultrasound, occurring post 14 weeks, involves the calculation of fetal weight obtained from measurements of foetal biometry. Foetal biometry measurements commonly include the measurement of the biparental diameter (BPD), head circumference (HC), abdominal circumference (AC), and femur diaphysis length (FL) which are used to estimate foetal birth weight (EFW) using a range of algorithms, with the most commonly referenced algorithms being M. J. Shepard and F. P. Hadlock formulae (18) (19). Foetal weight is the major parameter used to determine the foetus’ growth, which is compared to reference charts to determine risk of abnormalities based on the foetus’ size for its gestational growth. Thus, accuracy of the foetal weight is crucial for its well-being and outcomes, and in turn, accuracy of each parameter used for the calculation of EFW. An issue with this system is that terminology for foetal growth and size is quite varied, and some are used interchangeably (10), for example the terms for ‘small for gestational age’ (SGA) and ‘intrauterine growth restriction’ (IUGR). While both describe a suggested decrease of foetal growth rate, IUGR is not specified by EFW, while EFW is used to characterize SGA, which excludes developmental abnormalities. In contrast, IUGR relies on accurate measurement of the AC, with its sensitivity optimised to as high as 95% if the AC value is below the 2.5^th percentile (10). 

Quality Control in Foetal Biometry

Post 14-week foetal examinations are centred on the accurate measurement of foetal biometry, which is dependent on the operator’s recognition and measurement of prominent anatomic landmarks (1), therefore quality control processes are mostly focused on this area of examination. Expected-value bias is the most common error affecting foetal biometry measurements, particularly in third trimester measurements. This error occurs when the operator adjusts the estimated measurement, such as a circumference and length for the observed gestational age to match a previously calculated gestational age, resulting in a biased calculated foetal growth estimation (20). A study undertaken suggested within their study results, 91.4% of their saved standard biometric plane measurements were affected by expected-value bias, with calculated bias to affect proportions of  85.0% for HC, 92.9% for AC and 94.9% for FL measurements (20). 

The average foetus size has increased in recent decades (10), which prompted the need for quality control and revision of the method of measurement. A method of quality control addressing this error involves assessment of at minimum 10% of stored images taken by an individual sonographer, to analyse the z-score distribution of measured foetal parameters and verify calliper placement (19). Upon review, sonographers that produced poor quality images or record measurements consistently outside of a 95% limit of agreement or significant deviation from z-score expected values are advised retraining. Additionally, revision of foetal biometry charts is advocated to be limited to charts that are prospectively, truly population-based and derived from research with a ‘minimisation of methodological bias’ (19). Salomon, L. et al. ⁽¹⁹⁾ recommends that image review be undertaken by an experienced individual who is capable of understanding and implementing quality assurance and quality control adequately.

Accreditation and Operator Certification

Much of the literature recognise the importance of accreditation and operator certification to promote accurate measurements and image quality in obstetric ultrasound. In a study investigating factors involved in image quality, Gurava S (6) advised that early education is a major influence in the optimal and adequate application of ultrasound (3). Site accreditation is utilised to promote operator compliance with multiple processes such as; correct calibration of equipment, appropriate cleaning and sanitisation of transducers, health and safety regulations, correct storing and retentions of documents, reporting policies and adherence with quality assurance and quality control systems to enhance patient safety and quality of care (1). On the other hand, education in ultrasound practice refers to the certification of operators which denotes to compliance of established quality control programs ⁽¹⁾. In a study by Abuhamad et al. ⁽²²⁾ in the US, improvement measured by case study scores was demonstrated when healthcare sites actively sought accreditation and sonographers complied with quality assurance standards and guidelines in obstetric ultrasound examinations. In Australia, operators are required to complete an online educational course including an assessment of images with a tutor sonographer for nuchal translucency measurement certification (15). In addition, operators have access to multiple educational resources such as websites and newsletters and are expected to access self-auditing software such as ‘Fetal Medicine Foundation’, to allow monitoring of performance. Annual audits are also undertaken to analyse the distribution of an individual’s NT measurements and assess image quality based on a preset standard (15).

The method of educating, certification, and quality assurance is approved in national societies of obstetricians and gynaecologists in multiple countries namely, Australia, Austria, Cyprus, Germany, and Italy for NT measurement training (5). On the other hand, in comparison to continuous monitoring and regular evaluation of individual operators, Gabriel et al. (10) suggests this method’s influence on NT measurement accuracy is insignificant. In addition, several other research trials also uphold the view that education and accreditation, although required, is not a baseline to guarantee accuracy in measurements amongst sonographers (16) (18).

Imaging Improvements

Past literature reviews have highlighted the shortcomings of image quality standards in ultrasound, yet current standards remain largely unchanged ^{(1)(5) (6).} Improved screening may be possible using a combined approach that includes foetal biometry measurements and other additional clinical, biological, and imaging markers. It is proposed that imaging accuracy can be improved when the ‘biometric component’ is better standardised for all health professions that care for pregnant women ^(19).

Studies have investigated the standardisation of biometry measurements to produce better accuracy between health care centres ^(3). Other studies suggest the use of local standards is more accurate to each specific clinical site’s equipment ^(5). Standardisation exercises can be completed before ultrasound examinations to refresh the operators of measurement protocol and proceed to achieve more precise foetal measurements ^(8). Customised growth charts rather than population-based growth charts can also be considered, a notable example including customised estimated foetal weight centiles for prenatal ultrasound diagnosis of abnormal foetal growth. When comparing the population based EFW centiles to the customised EFW centiles, customised EFW centiles demonstrated distinctly higher rates of abnormal pregnancy outcomes compared to the population standard centiles ^{(23) (24).} However, there are mixed results with customised growth charts, some citing that customisation of the EFW did not strengthen the relationship between small for gestational age scores and neonatal morbidity ⁽²⁵⁾. Salomon, L. and Ville, Y ⁽¹⁾ contends that centre specific growth charts are acceptable for clinical use if differences can be confirmed to be caused from primarily from patient characteristics and variation of ultrasound equipment. The author claims that reference charts that vary due to measurement techniques would imply each operator will have to create their own reference range which would be impractical and time consuming ^(1).

Advancements in ultrasound technology has led to the possibility of three-dimensional ultrasound to increase interobserver and intraobserver variability. Studies suggest that the use of three-dimensional ultrasound by an inexperienced operator allows faster measurements than the two-dimensional ultrasound method and may facilitate the acquisition of higher-quality images for measurement of the AC ⁽¹⁾⁽⁷⁾. Improvements in resolution of grayscale ultrasound and recent application of three‐dimensional ultrasonography has allowed ease in evaluation of foetal fat and muscle components, an example including whole foetal limb‐volume measurements ^(1).  Studies have imposed that combining fractional limb volume with two-dimensional biometry improves the precision of EFW ^(25).  Furthermore, with the rapid development of US technology, studies have proposed the use of artificial intelligence (AI) to process larger quantities of data, increasing the consistency and accessibility of ultrasound. Benacerraf et al. ⁽⁴⁾ proposes the reduction of operator error through the development of smart tools to provide software with automatic recognition of normal and abnormal foetal anatomy by image scoring and analysis conducted by AI. Software that quantifies knowledge, psychomotor skills, and image interpretation would be an ideal method for characterizing the difference in operator skill level between novice and expert, specific to ultrasound operation ⁽⁴⁾.

The consistency and efficiency of three-dimensional automation of foetal measurements has been investigated, yet automation is still not routine practice ^(21).  Current automation techniques include segmentation approaches, which are algorithms that involving threading and edge detection on ultrasound images ^(26). Examples of current vector techniques and their uses include the gradient vector flow contour method for elliptical shape parameters (AC, HC, BPD, and NT measurements) and morphology-based techniques for the femur length of the foetus ^(26). Following the use of automation for feature extraction of the image, classification techniques (neural network and support vector machine) are implemented to predict foetal abnormalities. Future segmentation trends that are suggested for clinical use include the neural network-based approach for diagnostic aid. Neural network approaches include collection of raw patient data with several AI techniques are applied for classification or detection ^(26). An example of utilisation of the neural network is in the detection of the IUGR foetus and is called the variational level set method.  The introduction of diagnosis assisted by the segmentation process and neural network model aims to improve the accuracy, precision, and computational speed ^(26). An important issue in computerisation of US measurements is the minimisation of bias when training the computers to perform a task. The AI system learns from teaching by humans who may present their own biases to the learning process, creating in biased AI models ^(20). Further studies on automation accuracy are suggested to be based across multiple centres to reduce bias in selection bias or inconsistency presented by varied clinical practice differing from published guidelines ^(21).

Conclusions

The maintenance and monitoring of obstetric ultrasound measurements is crucial, as the impact of biometric measurement errors can lead to a misdiagnosis and possible complications for the fetus, therefore QA and QC systems are essential for a clinical setting. Many sources discussed the possibilities of inaccuracies in obstetric ultrasound measurements; however, few methods were proposed to address these inconsistencies, and few QA and QC systems are in place currently. Routine auditing is a commonly suggested system advocated by sources; however, these are regularly implemented amongst healthcare sites already to monitor operator performance and image quality (1). Beyond this, quality control in obstetric ultrasound is aimed towards preventing poor quality images and measurements, using artificial intelligence to amount for the inexperience and lack of resources in this region of imaging (4). In addition, there is an increased focus on accreditation and certification for operators as a means to improve image quality and guarantee accuracy, however, this has been demonstrated to not be a long term solution (18), as regulated maintenance of skills are required to prevent errors (10). Though there are multiple developments in obstetric ultrasound regarding feedback systems to increase efficiency and accuracy in diagnosis (21), there is a lack of literature covering proper implementation of these systems beyond the preliminary testing stages (26).

Despite a lack of literature addressing the implementation and methods of QA and QC processes within a clinical setting, there is the recognition of the importance of these systems in maintaining accurate measurements in obstetric ultrasound (1) (3). There is a great amount of focus on education and training in various countries (4) (5) to ensure sonographers are well trained and informed on the standard requirements of the procedures. The most recent articles (21) (26) demonstrate the possibility of many advancements in technology and strategies that are dedicated on maximising accuracy and image quality in obstetric ultrasound examinations.