Analysis of the Child Behaviour Checklist

Chapter II: Literature Review

As suggested in the introduction, numerous researchers have explored the prevalence of emotional and behavioural problems across the globe. Researchers have also investigated correlates (e.g., age and gender) associated with emotional and behavioural problems. The psychometric properties of instruments assessing emotional and behavioural problems have also been a subject of interest. In addition, researchers have also investigated cross-cultural similarities and disparities among emotional and behavioural problems. The extensive literature that addresses these issues, and which also helped formulate the rationale for the current study, is presented in five sections. The first section highlights the problems associated with epidemiological studies and compares the two main approaches to epidemiological studies, namely the categorical and the empirical approach. The second section provides a detailed description of the CBCL including the evolution of the measure, its psychometric properties, its advantages and disadvantages, as well as its range of applicability. The third section provides a description of the theoretical rationale for assessing cultural similarities and disparities associated with emotional and behavioural problems. Multicultural findings based on the CBCL as well as age and gender differences associated with emotional and behavioural problems are also reported. The fourth section consists of a review of the various processes involved in assessing the psychometric properties of instruments and findings based on psychometric properties of the various translations of the CBCL. The fifth section consists of a brief cultural and socio-political description of Pakistani society followed by a description of the salient features (i.e., family, community and cultural factors) in relation to emotional and behavioural problems in Pakistani society. Finally, there is a description of the objectives of the current study.

Epidemiology of Emotional and Behavioural Problems

Current reviews of epidemiological studies indicate that there is a high prevalence of emotional and behavioural problems among children and adolescents around the world (Costello et al., 2004; Hackett & Hackett, 1999; Waddell et al., 2002). In one review, Costello et al. compared findings across several developed countries (including Canada, the United States, the United Kingdom, Germany and Australia) to investigate the prevalence of emotional and behavioural problems as well as that of other psychological problems. Based on their findings, the overall prevalence rates of psychological problems among children and adolescents had a very broad range (0.1% to 42%), with varying rates for each category of disorder. Categories include disruptive behaviour disorders (i.e., conduct disorder, oppositional disorder and attention deficit hyperactivity disorder), mood disorders (i.e., major depressive disorder and bipolar disorder), anxiety disorders (i.e., phobias, generalized anxiety disorder, obsessive compulsive disorder, and post-traumatic stress disorder) as well as substance abuse and dependence. A critical examination of the studies included in the review revealed that variations in prevalence rates may be attributed to methodological flaws such as substantial disparity across studies with regard to sample size and the age range assessed. Moreover, differences across studies in terms of the measures used, the criteria employed as well as the type of informant may also have influenced the findings.

In contrast to Costello et al.’s (2004) review, Waddell et al.’s (2002) review was based on more stringent criteria; studies based on samples of similar size and age range, as well as using similar methodology were compared. Based on Waddell et al.’s review, the prevalence rates of emotional and behavioural problems varied between 10% and 20%. Although findings from both reviews vary considerably, the prevalence rates of emotional and behavioural problems across developed countries is still high and warrants serious attention. Moreover, methodological disparities across studies underscore the need for a uniform methodology to investigate the prevalence of emotional and behavioural problems.

In contrast to developed countries, there are few researchers investigating prevalence rates in developing countries (e.g., Bangladesh, India, Sri lanka, Sudan, and Uganda) (Costello, 2009: Fleitlich-Bilyk & Goodman, 2004; Mullick & Goodman, 2005; Nikapota, 1991; Prior, Virasinghe, & Smart, 2005). Moreover, there is a scarcity of reviews of the existing studies. In one review, Hackett and Hackett (1999) compared results from India, Puerto Rico, Malaysia and Sudan, and the prevalence rates of psychological disorders ranged from 1% to 49%. Similar to research in developed countries, researchers attribute variations in findings to methodological problems across studies, which include an inadequate sample size, paucity of explicit and internationally accepted diagnostic criteria, as well as inconsistencies in assessment procedures (Fleitlich-Bilyk & Goodman, 2004). Moreover, prevalence rates among developing countries may also partly be linked to the social, economic and medical environment. For example, lack of medical resources and awareness about psychological problems may result in parents not knowing how to seek help (Gadit, 2007). Social taboos further compound the problem, preventing people from reporting problems and deterring help-seeking behaviour (Samad, Hollis, Prince, & Goodman, 2005). More importantly, cultural variations in the conceptualization and identification of psychological problems may result in varied reporting of symptoms (Gadit, 2007). These environmental differences and methodological inconsistencies across studies emphasize the need for a cross-culturally robust methodology to investigate the prevalence of emotional and behavioural problems.

Along with methodological problems and environmental differences, emotional and behavioural problems merit investigation because they affect multiple aspects of children’s functioning such as academic performance and social adjustment (Montague et al., 2005; Nelson et al., 2004; Vitaro et al., 2005). Researchers also state that there is high comorbidity among emotional and behavioural problems, (SteinHausen, Metze, Meier, & Kannenberg, 1998) which creates multiple problems for children and their caregivers. Moreover, many childhood disorders continue and influence functioning during adulthood. In fact, many adult disorders are now recognized as having roots in childhood vulnerabilities (Maughan & Kim-Cohen, 2005; Tremblay et al., 2005). Furthermore, recognizing and treating problems early can reduce the burden of the enormous human and financial costs associated with the assessment and intervention, especially in countries where resources are scarce (Costello, Egger, & Angold, 2005; James et al., 2002; Waddell et al., 2002). In addition, cross-cultural epidemiology of children’s emotional and behavioural problems may also better inform current knowledge about the characteristics, course, and correlates of such problems, which in turn provide a scientific basis for appropriate mental health planning (Achenbach & Rescorla, 2007; Waddell et al.). Therefore, there is a strong need for a methodology that can be utilized for clinical as well as research purposes to assess emotional and behavioural problems among children and adolescents across cultures.

Current literature indicates that there are two main approaches to investigate the epidemiology of emotional and behavioural problems, namely the categorical and the empirical approach. There are several differences in both approaches including conceptualization of psychological problems as well as the methodology employed for their assessment. Both approaches will be discussed briefly.

The categorical approach. The categorical approach, based on the biomedical perspective, views psychological problems as a group of maladaptive and distressing behaviours, emotions and thoughts which are qualitatively different from the typical (Cullinan, 2004). That is, similar to medical diseases, an individual may or may not have a specific psychological disorder. Traditional epidemiological studies are based on the categorical approach as embodied in various editions of the Diagnostic and Statistical Manual for Mental Disorders (DSM) (American Psychiatric Association (APA), 1980; 1987; 1994; 2000) and the International Classification of Diseases (WHO, 1978; 1992). Examples of instruments used in traditional epidemiological studies to derive DSM diagnoses include the Diagnostic Interview Schedule for Children (DISC) (Costello, Edelbrock, Kalas, Kessler, & Klaric, 1982) and the children’s version of the Schedule for Affective Disorders and Schizophrenia (Kiddie-SADS) (Puig-Antich & Chambers, 1978). At present, there is considerable debate about the validity of epidemiological studies based on the categorical approach. Researchers have highlighted that inconsistencies in prevalence rates may be due to conceptual and methodological issues linked with the DSM as well as methodological disparities among studies (Achenbach & Rescorla, 2007; Waddell et al., 2002). Each of these factors will be discussed briefly.

DSM related problems. Multiple conceptual and methodological problems are associated with the DSM. First, the DSM does not provide a methodology to operationally define different psychological disorders (Widiger & Clark, 2000). To operationally define DSM criteria, various diagnostic interviews such as the DISC have been developed. Unfortunately, meta-analyses indicate that the diagnoses based on the DISC and other diagnostic interviews are not in agreement with diagnoses made through comprehensive clinical interviews, which indicate that, neither diagnostic nor clinical interviews provide good validity criteria for testing DSM categories (Achenbach, 2005; Costello et al., 2005; Lewczyk et al., 2003). Second, the diagnostic categories and criteria provided in the DSM continue to change as reflected in the changes across the various editions of the DSM, namely the third edition (APA, 1980), third edition revised (APA, 1987), fourth edition (APA, 1994), and fourth edition text revised (APA, 2000), making comparisons across editions problematic (Achenbach, 2005). Third, although the current version, known as the DSM-IV-text revised (APA, 2000), aims at introducing cultural sensitivity in assessment and diagnoses by including an “outline for cultural formulation and a glossary of culture-bound syndromes” (APA., 2000, pg. 897), it does not provide criteria or guidelines regarding the use of the classification system with specific cultural groups (Paniagua, 2005). Since many of the DSM diagnostic criteria are based on Euro-American social norms, it is difficult to use the DSM criteria to identify psychopathology in individuals from other cultures.

In addition, there is growing consensus among researchers that DSM categories need to be more appropriate for children and adolescents of different ages and gender (Doucette, 2002; Segal & Coolidge, 2001). Turk et al. (2007) also highlight the saliency of factors such as age and gender when investigating prevalence rates. However, at present, this is not the case. Costello et al. (2005) have stated that the constant developmental changes of childhood create the need for an age- and gender- specific approach to epidemiology.

Before incorporating a developmental perspective in epidemiological studies, it is essential to have a better understanding of developmental psychopathology. Developmental psychopathology is based on the view that problems arise from different causes, manifest themselves differently at each stage, and may have diverse outcomes. Developmental psychologists do not support a specific theory to explain all developmental issues. Instead, they try to incorporate knowledge from multiple disciplines (Cicchetti & Dawson, 2002). Moreover, developmental psychopathology also includes an analysis of the existing risk and protective factors within the individual and also in his/her environment over the course of development (Cicchetti & Walker, 2003).

According to Costello and colleagues (2004), a developmental perspective in epidemiological studies is based on the inclusion of certain principles. First, precise assessment measures for the different phases in childhood and adolescence are required to compare children’s functioning with that of their same-age peers. For example, problems such as fear of dark places is considered typical for 6-year-olds but not for 12-year-olds. Furthermore, the developmental perspective would include longitudinal studies to evaluate the ways in which developmental processes influence the risk of specific psychological disorders. For example, the developmental trajectory of physical aggression is such that there is an increase in Aggressive Behavior during the first few years of childhood, but it progressively decreases until adulthood (Tremblay et al., 2004). Moreover, developmental epidemiology would include frequent assessments to determine the onset of disorders. Frequent assessments would also assist in the identification of environmental and individual factors that contribute to the development of psychopathology. Although the developmental perspective emphasises the need for age- and gender-specific diagnostic criteria, longitudinal studies as well as frequent assessments, it is difficult to incorporate this perspective in studies based on the categorical approach as it is not sensitive to developmental changes.

Methodological disparities. A critical analysis of categorically based epidemiological studies reveals multiple methodological problems. These include inconsistencies in assessment and sampling procedures as well as absence of guidelines about using data from multiple sources. In terms of assessment procedures, both symptoms as well as significant impairment are required to identify children with disorders. This is corroborated by Costello et al. (2004), who report that the disparity in the prevalence rates of phobias (i.e., 0.1% to 21.9%) may be attributed to how phobias were assessed in each study, in particular, whether both symptoms (e.g., fear of open places, snakes) as well as significant functional impairment were taken into account in the identification of phobias. Waddell et al. (2002) state that the use of standardized measures has lead to an improvement in the assessment of symptoms; however, problems still exist with regard to how impairment is gauged or how measures may be combined to include symptoms as well as impairment. Another problem with assessment procedures is that different interview schedules (e.g., DISC and the Kiddie-SADS) and DSM editions have been used across studies, which may have contributed to differences in prevalence rates.

Incompatible sampling procedures may also have led to disparities in overall prevalence rates in categorically based epidemiological studies (Waddell et al., 2002). For example, studies such as the Great Smokey Mountains study (Costello, Angold, Burns, Erkanli, Stangel & Tweed, 1996) were relatively more comprehensive, and investigated a larger number of diagnostic categories than other studies. As a result, higher overall prevalence rates of psychological problems were reported compared to studies that did not assess as many disorders. Another sampling issue is that reviews were based on studies that differed with regard to the age range assessed; some studies focused on a younger age bracket (i.e., between 8 to 11 year olds), others on an older age bracket (i.e., 11 years and older), whereas some researches included a very broad age range (i.e., 6 to 17 year olds). In addition, there were inconsistencies across studies in terms of the type of informant used; some studies relied on parents only, some on children, while some combined data from parents, children as well as teachers. Differences in the age brackets assessed as well as the use of different informants may have contributed to disparities in epidemiological findings.

Another salient issue with regard to categorically based epidemiological studies concerns the coordination and interpretation of information from multiple informants. Since problem behaviours may only occur in specific situations or with specific individuals, multiple informants (e.g., teachers, parents and children) are necessary. However, since the respondent’s context and perception have a great impact on the identification of psychological problems, poor agreement among respondents is frequently reported. For example, children normally report higher rates of internalizing symptoms (e.g., anxiety and depression) while parents tends to report higher rates of externalizing symptoms (e.g., Conduct Problems) (Rubio-Stipec, Fitzmaurice, Murphy, & Walker, 2003). Additionally, children are not considered reliable reporters of their own behaviour due to differences in cognitive abilities as well as the ability to report their own behaviour (Achenbach & McConaughy, 2003). Despite such findings, the categorical approach does not provide guidelines regarding obtaining and interpreting data from multiple sources, which complicates matters in terms of how to combine data into yes-or-no decisions about different symptoms.

The various conceptual problems associated with the DSM as well as the methodological flaws in epidemiological reviews highlight the problems associated with using the categorical approach as a basis for epidemiological studies. Moreover, these issues underscore the need for an approach that is methodologically sound and culturally appropriate for cross-cultural comparisons. An alternative to problems linked to the categorical approach, where an a priori criterion is imposed, can be a system that is empirically based and identifies problems as they occur in a population. Such an approach would be helpful in highlighting cultural differences in the manifestation of different emotional and behavioural problems. Moreover, there is also a need for a methodology that can be employed in a standardized, systematic fashion. Although the empirical approach is not a panacea for problems associated with epidemiological studies, it does provide solutions to some of the types of errors in the categorical system.

Empirical or dimensional approach. The empirical or dimensional approach, in accordance with a psychosocial perspective, views mental health as a continuum. The dimensional perspective supports the notion that all individuals experience problems involving behaviours, emotions and thoughts to varying extents. Those who experience such problems to an extreme extent (unusual frequency, duration, intensity, or other aspects) are more likely to have a psychological disorder (Cullinan, 2004). In contrast to imposing a priori criteria on children’s emotional and behavioural problems, the empirical approach identifies problems as they present themselves in the population. According to Cullinan (2004), there are certain steps involved in developing a dimensional classification system for emotional and behavioural problems. These steps include creating a collection of items that reflect measurable problem behaviours experienced by children, identifying a group of children to be studied, assessing every child in the group on each problem, and investigating the data to identify items that co-vary, thus leading to the identification of different dimensions or factors. After the dimensions have been derived, the pool of items can be used to assess and classify emotional and behaviour problems among new populations. Given that the empirical approach is based on the identification of co-occurring problem behaviours in the population, instead of imposing a priori criteria, it is a favourable approach for cross-cultural epidemiological studies.

Within empirical approaches, the Achenbach System of Empirically Based Assessment (ASEBA) provides a good framework for epidemiological studies for multiple reasons. First, being empirically based, ASEBA identifies emotional and behavioural problems as they occur in the population. Second, it is based on a developmental perspective, has a uniform methodology, and also provides explicit guidelines about using data from multiple sources (Achenbach & McConaughy, 1997; Achenbach & Rescorla, 2001). Hence it provides solutions to problems that arise in the categorical approach. Moreover, Cullinan (2004) and Krol et al. (2006) state that ASEBA measures have been used more extensively compared to other measures of emotional and behavioural problems, such as the Conners Rating Scale- Revised (Conners,1990) and the Strengths and Difficulties Questionnaire (Goodman, 1997). Achenbach system of empirically based assessment (ASEBA).

Although the ASEBA has a non-theoretical, empirical base per se, it is greatly influenced by the principles of developmental psychopathology. For example, Achenbach highlights that problems may include thoughts, behaviours, and emotions that may manifest themselves differently depending on the age and gender of the individual (Greenbaum et al., 2004). Therefore, each ASEBA form provides norms based on the age and gender of the child, which enables an individual’s functioning to be assessed in comparison to same-age peers. Furthermore, ASEBA is a multiaxial system that encompasses a family of standardized instruments for the assessment of behavioural and emotional problems as well as adaptive functioning. The five axes of the assessment model include parent (Axis I) and teacher (Axis II) reports, cognitive (Axis III) and physical (Axis IV) assessments as well as the direct assessment of children (Axis V) (Achenbach & McConaughy, 2003). The use of different ASEBA instruments provides a standardized and uniform methodology to incorporate information from multiple sources.

Furthermore, all ASEBA instruments are empirically based. In accordance with the empirical approach, the construction of the ASEBA forms involved a series of steps (Achenbach & McConaughy, 2003). Initially, a collection of potential symptom behaviours (i.e., items) was derived from multiple sources. These items were operationally defined in such a manner that respondents not trained in psychological theory could use them. In accordance with general item-development procedures, pilot tests were conducted to evaluate the clarity of items, response scales and item distribution. Finally, items that could differentiate between individuals who were not functioning well and their well functioning same-age peers were retained. Multivariate statistical analyses were applied to the retained items in order to identify syndromes of problems that co-occur. Syndromes were identified purely on the basis of co-occurrence, without any link to a particular cause. Subsequently, the syndromes of co-occurring problem items were used to construct scales. These scales were used to assess individuals in order to assess the degree to which they exhibit each syndrome. Since all ASEBA instruments are empirically based, findings can be compared on the basis of the manifestation of different emotional and behavioural problems, thereby providing a clearer picture of cross-cultural similarities and disparities of different emotional and behavioural problems.

In terms of the historical evolution of the system, ASEBA originated to provide a more differentiated assessment of child and adolescent psychopathology than the DSM. When ASEBA was developed, the first edition of the DSM (APA, 1952) had only two categories for childhood disorders, which included adjustment reactions of childhood and schizophrenic reaction childhood type (Achenbach & Rescorla, 2006). In contrast to the DSM, the first ASEBA publication highlighted more syndromes of emotional and behavioural problems (APA, 1952). Moreover, based on factor analyses, Achenbach (1966) identified two broad groupings of problems for which he coined the terms “Internalizing” and “Externalizing.” As described earlier, Internalizing Problems included problems with the self, such as anxiety, depression, withdrawal, and Somatic Complaints, without any apparent physical cause. On the other hand, Externalizing Problems included problems with other people, as well as problems linked to non-conformance to social norms and mores, such as aggressive and delinquent behaviour. Although all ASEBA forms are used extensively in clinical and research environments, the Child Behavior Checklist is the most widely recognized measure for the assessment of emotional and behavioural problems (Greenbaum et al., 2004; Webber & Plotts, 2008).

Child Behavior Checklist

An essential part and the cornerstone of Achenbach’s multiaxial, empirical system is the Child Behavior Checklist (CBCL). Although the CBCL assesses social competencies as well as problem behaviours, it is widely recognized as a measure of emotional and behavioural problems as opposed to social competencies. In fact, researchers suggest that the CBCL is the most extensively utilized measure for the assessment of problem behaviours among children and adolescents as observed by their parents and caregivers (Krol et al., 2006; Greenbaum et al., 2004).

Although there have been multiple revisions to the initial CBCL, all versions have the same format and consist of two distinct sections. The first section measures social competencies. Parents are asked to respond to 20 questions regarding the child’s functioning in sports, miscellaneous activities, organizations, jobs and chores, and friendships. Items also cover the child’s relations with significant others, how well the child plays and works alone, as well as his/her functioning at school. Finally, respondents describe any known illnesses or disabilities, the issues that concern them the most about the child, and the best things about the child (Achenbach & Rescorla, 2006). The second section assesses problem behaviour and consists of 118 items that describe specific emotional and behavioural problems, along with two open-ended items for reporting additional problems. Examples of problem items include “acts too young for age”, “cruel to animals”, “too fearful or anxious”, and “unhappy, sad or depressed”. Problem behaviours are organized in a hierarchical factor structure that consists of eight correlated first-order or narrowband syndromes, two correlated second-order or broadband factors (i.e., Internalizing and Externalizing Problems) and an overall Total Problems factor. Parents/caregivers are asked to rate the child with regard to how true each item is at the time of assessment or within the past 6 months. The following scale is used: 0 = not true (as far as you know), 1 = somewhat or sometimes true, and 2 = very true or often true. In the case of respondents with poor reading skills, a non-clinically trained clincian can also admisnter the CBCL (Achenbach & Rescorla, 2006). For respondents who cannot read English but can read another language, translations are available in over 85 languages (Berube & Achenbach, 2008).

Development of the CBCL.

The first version of the CBCL dates back to 1983. To date, there have been two revisions of the CBCL; the first one in 1991 followed by the second in 2001, leading to considerable improvements in the measure. The main weakness of the initial CBCL was that comparisons across different age groups and respondents were problematic since syndromes had the same names but different items across different age forms (i.e., 4 to 5, 6 to 11, 12 to 16 years) as well as across different respondent forms (i.e., CBCL, teacher report form [TRF], and the youth self report [YSR]) To rectify the problem, the 1991 version included two new types of syndromes, the core and cross-informant syndromes. Core syndromes represented items that clustered together consistently across age and gender groupings on a single instrument. Cross-informant syndromes were based on those items from the core syndromes that appear on at least two of the three different respondent forms (i.e., CBCL, TRF, and YSR) (Greenbaum et al., 2004). These revisions facilitated comparisons across different age groups and informants. Moreover, the 1991 version of the CBCL also had new national level norms, which included norms for seventeen and eighteen year olds. Apart from practical benefits, changes such as a broader age range and precise criteria for different developmental levels, genders and type of respondents, helped make the CBCL and ASEBA instruments more accurately representative of the developmental perspective of child psychopathology (Greenbaum et al.).

Achenbach (1991) also conducted exploratory principal factor analyses of the syndrome scales. Based on the loadings of different syndromes, Achenbach identified Anxious/Depressed, Withdrawn, and Somatic Complaints as indicators of Internalizing Problems, whereas Aggressive and Delinquent Behavior were identified as indicators of Externalizing Problems. Since Social Problems, Thought Problems and Attention Problems did not load consistently on either second-order factor, they were not placed in any group (Achenbach, 1991; Greenbaum et al., 2004). Although Internalizing and Externalizing Problems identify different types of behaviour, the two categories are not mutually exclusive and may co-occur within the same individual. This is supported by research findings that indicate that there was a correlation between the two groups in both clinic-referred (.54) and non-referred (.59) samples matched on the basis of age, sex, race, and income (Achenbach, 1991).

Description of the current CBCL.

The current CBCL was published in 2001 and covers ages 6 to 18 years (CBCL/6-18; Achenbach & Rescorla, 2001). The CBCL/6-18 (Achenbach & Rescorla, 2001) provides raw scores, T- scores and percentiles for the following: (1) the three competence scales (Activities, Social, School); (2) the Total Competence scale; (3) the eight cross-informant syndromes; (4) Internalizing and Externalizing Problems and (5) Total Problems. The cross-informant syndromes of the CBCL/6-18 include Aggressive Behavior, Anxious/Depressed, Attention Problems, Rule-Breaking Behavior, Social Problems, Somatic Complaints, Thought Problems, and Withdrawn/Depressed.

As far as similarities and differences from previous versions are concerned, the current CBCL introduced some major and a few minor changes. One major change was the introduction of the DSM-oriented scales, based on which CBCL and other ASEBA forms can now be scored in terms of scales that are oriented toward categories of the fourth edition of the DSM (A.P.A., 1994). The introduction of the DSM-oriented scales has combined the categorical and empirical approaches and enables users to view problems in both the categorical and dimensional approaches (Achenbach, Dumenci & Rescorla, 2003; Achenbach & Rescorla, 2006). The DSM-oriented scales include six categories, namely Affective Problems, Anxiety Problems, Somatic Problems, Attention Deficit/Hyperactivity problems, Oppositional Defiant Problems as well as Conduct Problems. These scales are based on problem items that mental health experts from sixteen cultures across the world rated as being consistent with particular DSM diagnostic categories. Similar to the empirically based syndromes, the DSM- oriented scales also have age-, gender- and respondent-specific norms.

Another major change was that new normative data was collected using multistage probability sampling in forty U.S. states as well as the District of Columbia. The selected homes were considered to be representative of the continental United States with respect to geographical region, socio-economic status, ethnicity and urbanization (Achenbach & Rescorla, 2001). Moreover, complex new analyses based on new clinical and normative samples were conducted. However, the eight syndromes and Internalizing and Externalizing groupings published in 1991 were replicated with minor changes. Research findings indicated that correlations between scores on the 1991 syndromes and their 2001 counterparts ranged from .87 to 1.00 (Achenbach & Rescorla, 2001