KEEP Intervention: Reducing Behaviors Problems in Foster Toddlers

Abstract

Infants and toddlers are the fastest growing population in the child welfare system. Externalizing behavior problems are highly prevalent among children in foster care, placing them at risk for placement disruptions and later personal and social maladjustment. Prior research indicates that the KEEP intervention is effective in reducing child externalizing behavior problems. Moreover, the KEEP foster parent intervention was recently implemented in a foster parent population and yielded consistent results (i.e., the intervention was found to be effective in reducing child problem behavior). The aim of this study was to examine how the KEEP intervention affects parenting self-efficacy and levels of externalizing problem behavior of toddlers in foster households. It was hypothesized that foster caregivers receiving the KEEP intervention would report lower levels of externalizing problem behavior from toddlers placed in their home by the foster system. Moreover, parenting self-efficacy would increase. Foster caregivers (n = 100) of toddlers (n = 100), ages 18 months to 36 months, that were entering their first out-of-home placement will be recruited for a year long, pretest/posttest, randomized-control trial of the KEEP intervention.

Keywords: parenting, foster, toddler, self-efficacy, KEEP, intervention

The environmental status quo for infants and toddlers in the United States has become rather grim. Rates of child maltreatment are increasing for children three years and younger, while rates are decreasing in other age groups (USDHHS, ACF, ACYF, 2013). Additionally, more children die of abuse and neglect in the first year of life than in any other year of childhood (Child Welfare Information Gateway, 2016). In response to the elevated risks that young children are encountering, the child welfare system in the US has made the physical safety of infants and toddlers a primary aim. Foster care is Child Welfare’s primary intervention to ensure the safety of children who have or are currently experiencing life-threatening forms of abuse and/or neglect. As such, infants and toddlers are the fastest growing population in the child welfare and foster care systems. And it does not seem to be helping.

There are now more than 124,500 children, five years and younger, in the United States’ foster care system, and next year another 15,000 young children are predicted to be part of that statistic (Child Welfare Information Gateway, 2016). Moreover, although foster care can interrupt abuse and neglect by providing children with a temporary, safe out-of-home placement setting, it is far from a perfect solution. Children in foster care are exposed to a variety of risks that are strongly linked to long-term deficits in emotional, behavioral, cognitive, and social functioning  (e.g., Cook, 1994; Festinger, 1983; Kufeldt et al., 1989; Millham et al., 1986; Pardeck, 1984; Penzerro & Lein, 1995). One of the primary aims of foster care is to find a safe permanent placement for the child as quickly as possible. Unfortunately, the length of stay for a child in the foster system is approximately two years, and the longer it takes for a child gain permanency, the more susceptible they are to compounding effects that further reduce the likelihood of gaining permanency (Newton, Litrownik, & Landsverk, 2000; Rubin, O’Reilly, Luan, & Localio, 2007).

Previous research indicates that placement disruptions (e.g., moving from foster home to foster home) drastically reduce the chances of achieving permanency (Fisher et al., 2005). Furthermore, placement disruptions are linked with numerous negative outcomes (Harden, 2004; Herrenkohl, Herrenkohl, & Egolf, 2003; Newton et al., 2000). Children with a large number of placement changes are at higher risk for running away from placements, spending greater time in residential facilities, and involvement with other service sectors, like juvenile justice (Harden, 2004; James, 2004; James, Landsverk, Slymen, & Leslie, 2004; Jonson-Reid & Barth, 2000).

One of the most frequently cited explanations for a placement disruption is the inability of the foster parents to manage child behavior problems (Holland & Gorey 2004; James 2004; Brown & Bednar 2006). Supporting this finding is evidence of a linkage between behavior problems and changes in foster care placements. Numerous findings indicate that externalizing behavioral problems are associated with higher levels of placement change (Chamberlain et al., 2006; Newton, Litrownik, & Landsverk, 2000), with behavior-related placement changes accounting for a significantly large portion of all placement changes (James, 2004). While investigating the relationship between externalizing behavior problems and placement disruptions, Chamberlain et al. (2006) found that the day-to-day occurrences of common child behavior problems predicted significant increases in risk for foster placement disruptions over a 1 year period. As evidence by the findings, externalizing behavior problems are a pervasive issue that contributes to placement disruptions and long-term negative outcomes for children within the child welfare system. Thus, it is critical to examine empirically supported interventions that effectively address the pervasiveness of externalizing behavior problems in the foster system.

Theoretical model

Over the past two decades, research has consistently linked externalizing behavior problems with ineffective parenting practices (Gelfand & Teti 1990; Laub and Sampson 1988). Interventions that focus on teaching and supporting parents to use more effective parenting methods have emerged as a mainstay of empirically grounded intervention and prevention efforts. Interventions have typically targeted specific parenting practices that operate as either protective or risk factors. Protective factors include practices such as parental reinforcement and positive mentoring (Catalano et al. 2004; Eddy & Chamberlain, 2000). Risk factors include the use of harsh or overly lax discipline, inadvertent reinforcement of coercive behaviors, and lack of involvement (Patterson 1982). Improvement in parenting practices has been shown to decrease child behavior problems (Eddy et al., 2005; Patterson et al., 2004).

Current practices

US federal policy requires that prospective foster caregivers be trained in the appropriate knowledge and skills in order to meet the needs of the child in their care, and, if necessary, training is to continue after placement (Foster Care Independence Act of 1999, H. R. 3443). This policy only provides general guidelines for training, and it does not specify the implementation procedures. Accordingly, nationwide there is incredible variation in the type of pre-service training offered, the duration of the training, and the requirements for continued in-service training (Grimm, 2003). Overall, it is unlikely that foster caregivers are provided with adequate information and instruction on challenging behavior management techniques. Moreover, it is unlikely that they would get to practice or feedback on their attempts to apply their training to their ongoing interactions with the children in their care. Lastly, very little research has been conducted on the effectiveness of these programs in actually reducing behavior problems in children in foster care, so we must look to the current state of affairs.

Intervention

Keeping Foster Parents Trained and Supported (KEEP) is a group-based intervention that targets providing foster parents with skills to more effectively address child behavioral challenges as well as information about normative developmental expectations that may change foster parents’ perceptions of children’s behavior. The KEEP intervention originates from a social-learning focused parent-training interventions (Hurlburt, Barth, Leslie, Landsverk, & McCrae, 2007) such as Parent Management Training (Kazdin, Esveldt-Dawson, French, & Unis, 1987; Patterson, Chamberlain, & Reid, 1982), Parent Child Interaction Therapy (Eyberg et al., 2001), the Incredible Years (Webster- Stratton, 1998), and is adapted from Multidimensional Treatment Foster Care (Chamberlain, Moreland, & Reid, 1992; Fisher & Chamberlain, 2000). KEEP has been tested specifically with foster parents and resulted in reduced child behavioral problems, increased likelihood of positive placement changes (e.g., reunification), and reduction in foster home placement changes among children with high numbers of prior placements (Chamberlain, Leve, & DeGarmo, 2007; Price et al., 2008).

Parenting self-efficacy

Parenting self-efficacy (PSE) can be defined as the degree to which parents expect to competently and effectively perform their roles as parents (Teti & Gelfand, 1991); it is rooted in general self-efficacy theory. Guided by social learning theory, general self-efficacy refers to the belief in one’s ability to perform behaviors successfully (Bandura, 1977). Overall, self-efficacy includes the motivation, cognitive resources, and courses of action necessary to implement control over a specific task or event (Ozer & Bandura, 1990). In general, high levels of self-efficacy has been found to predict competence in the face of environmental demands, conceptualize difficult situations as challenges, have less negative emotional arousal in the face of stress, and exhibit perseverance when challenged (Jerusalem & Mittag, 1995). In contrast, low self-efficacy is associated with self-doubt, high levels of anxiety when faced with adversity, assuming more responsibility for task failure than success, interpreting challenges as threats, and avoiding difficult tasks.

Based on PSE’s theoretical and empirical links to several dimensions of parenting behavior (Bor & Sanders, 2004; Coleman et al., 2002), it is logical to consider its associations with problem behavior during early childhood. In fact, PSE has been linked to children’s development in terms of behavioral adjustment (Bor & Sanders, 2004). For example, as early as 5 months, PSE was found to be positively related to concurrent ratings of infant soothability (Leerkes & Crockenberg, 2002). At age 2, Raver and Leadbeater (1999) found that PSE was inversely related to children’s concurrent difficult temperament among a sample of urban impoverished families. In a demographically similar sample of mothers with school-aged children, higher PSE was concurrently associated with less emotionally reactive and more sociable behavior (Coleman & Karraker, 2000). In terms of problem behavior, lower levels of PSE among mothers of preschool-aged children at high risk for developing conduct PSE has been found to mediate associations between parenting and children’s developmental outcomes (Coleman & Karraker, 2003).

Current Study

The primary aim of this study is to assess the efficacy of the KEEP intervention on decreasing externalizing behavior problems for toddlers in out-of-home foster placements. Along with the primary aim, this study is designed to examine positive parenting skills and parenting self-efficacy as products of the KEEP intervention that double as mechanisms of change. The current study will use a randomized, between-groups, repeated measures (i.e., pretest-posttest), control group design to assess the effect of the KEEP intervention on toddler externalizing behavior problems. The categorical independent variable (IV) will be random assignment to the treatment condition (i.e., KEEP intervention or services-as-usual). The continuous quantitative dependent variable (DV) will be toddler externalizing behavior problems. Positive parenting skills and parenting self-efficacy will both be continuous quantitative DVs.

Research questions and hypotheses

The current study will address the following research questions: (1) Does receiving the KEEP intervention affect foster caregivers’ use of positive parenting behaviors? Based on previous research indicating a relationship between receiving the KEEP intervention and an increase in positive parenting behaviors (e.g., Chamberlain et al., 2008), I hypothesize that foster parents who receive the KEEP intervention will report an increased use of positive parenting behaviors. (2) Does receiving the KEEP intervention affect parenting self-efficacy? Along with the research supporting the previous hypothesis, previous research indicates that the acquisition of positive parenting behaviors is associated with increased parenting self-efficacy (Ardelt & Eccles, 2001; Izzo et al., 2000), so I hypothesize that foster parents who received the KEEP intervention will report higher levels of parenting self-efficacy than foster parents who did not receive the KEEP intervention. (3) Will foster caregivers who receive the KEEP intervention report a different amount of toddler externalizing behavior problems than those foster caregivers who do not receive the KEEP intervention? Based on prior research (e.g., Fisher, Burraston, & Pears, 2005), I hypothesize that foster parents who receive the KEEP intervention will report lower levels of toddler externalizing behavior problems than foster parents who do not receive the KEEP intervention.

Methods

Recruitment

We will recruit children between the ages of 18 months and 36 months who are entering new foster care placements in Multnomah County (Oregon, USA). The Multnomah County Department of Human Services Child, Adult, and Families Division will refer eligible children to us, and we will invite the foster parents of the child’s new placement to participate in the study. Over the course of the study, we expect some children to reunify with their biological parents or transition to a new foster care or adoptive placements. We will exclude (or defer recruitment of) those children who are expected to transition within three months post-placement. Also, if the child experiences any kind of unexpected placement change (e.g., transition to a new foster care, reunification, adoption) after completion of the baseline assessments, we will recruit the child’s new caregiver to participate in the study. We will continuously recruit for four years.

Participants

Participants will include at least 100 foster parent-child dyads (i.e., children newly placed in foster care and their respective foster parents). Children are eligible for the study if: (a) they are between the ages 18 months and 36 months at the time of baseline assessment, (b) they will most likely remain in the new foster placement longer than three months, (c) they are not considered to be “medically fragile” (i.e., not severely physically or mentally handicapped), and (d) their foster parents meet the inclusion criteria. Foster parents will meet inclusion criteria for the study if: (a) they are a certified foster parent in Oregon, (b) child welfare has newly placed a child between the ages of 18 months to 36 months in their care, (c) they speak English fluently, and (d) they have not previously participated in the current study. Participants will be randomly assigned, at an overall ratio of 1:1, to the intervention group or control group. Research assistants and the researchers will be blind to the group assignment.

Procedure

All procedures proposed in this study will be approved by the University of Oregon and State of Oregon Department of Human Services (DHS) Institutional Review Boards (IRB). Recruitment will be ongoing for two years, starting January 1^st, 2019 and concluding January 1^st, 2021. The study will continue for six-months after the final recruitment; all post-assessments will be completed prior to July 1st, 2021. During on-going recruitment, the Multnomah County Department of Human Services Child, Adult, and Families Division will refer children that meet inclusion criteria to us and provide us with their future foster parent’s contact information. A trained research assistant (RA) with a Master’s degree will contact the referred child’s future foster parent and determine if the foster parent meets inclusion criteria. When foster parent-child dyads meet criteria, research assistants will explain informed consent and obtain verbal and written consent to participate. Informed consent will include information about strict confidentiality procedures. If the foster household has more than one eligible foster parent, the family designates one foster parent to be the “primary caregiver.” The primary caregiver will be asked to participate in the KEEP intervention/services-as-usual groups and study assessments. In all cases, participant ID’s will be assigned to ensure confidentiality. During that same contact, the RA will schedule a home visit to conduct the baseline assessment.

Within one month of the child’s new placement, an RA will visit the foster home. During that visit, parent-child dyads will be asked to participate in a baseline assessment (i.e., pretest). The baseline assessment will take less than one hour and consist of a series of demographic questionnaires and parent-report measures. Along with the home visit, the baseline assessment includes over the phone administration of a parent-report measure. Participants will be compensated $50 for completing the baseline assessment. Then, participants will continue in the study according to their randomly assigned treatment condition.

Six months after the baseline assessment, a RA will schedule a home visit, and parent-child dyads will be asked to participate in the final assessment (i.e., posttest). The final assessment will take less than one hour and consist of the same demographic questionnaires and parent-report measures. The final assessment also includes over the phone administration of a parent-report measure. Participants will be compensated $75 for completing the final assessment; compensation for the final assessment is slightly greater than compensation for the baseline assessment to incentivize continued participation in the study. Since recruitment will be ongoing, participants will be designated to a particular Wave. All Waves will be six months long and include: equal intervention and control participants, baseline assessment within one month of the child’s placement, intervention group or services-as-usual, and a final assessment six-months after the baseline assessment. As detailed below, the intervention group will run every two months to maintain this timeline and intervention dosage equality across Waves. After completion of the final Wave, participants will be debriefed about the study, and foster parents that were assigned to the control condition will have the opportunity to participate in the intervention.

Intervention model

Participants will be randomly assigned, at an overall ratio of 1:1, to the intervention or services-as-usual condition. Participants randomly assigned to the intervention condition will receive KEEP parent-training in a group setting. Participants will attend a two-hour parenting group session once a week for 12 weeks. Groups will have 3 to 10 participants that are led by a trained facilitator. During the sessions, participants will receive parent training, supervision, and support in behavior management methods. The primary focus of the KEEP intervention is on increasing use of positive reinforcement, consistent use of non-harsh discipline methods (e.g., brief time-outs or privilege removal over short time spans), and management of “extra-tough” behaviors. Moreover, strategies for avoiding power struggles, managing stress, and engaging in developmentally appropriate play with toddlers are also included. Sessions are structured so that the curriculum content is integrated into group discussions and primary concepts are illustrated via role-plays. Home practice assignments related to the topics covered in class are given in order to assist parents in implementing the behavioral procedures taught in the group meeting. If participants miss a parent-training session, the material will be delivered during a home visit. Home visits have been found to be an effective means of increasing the dosage of the intervention for families who miss interventions sessions (Reid & Eddy, 1997).

Parenting groups will be formed based on dates of recruitment and completion of baseline assessments. Parenting groups will be conducted in community recreation centers, schools, or UO facilities. Several strategies will be used to maintain parent involvement, including: (a) provision of childcare–using qualified individuals so that parents will be able to bring children and know that their children will receive adequate care, (b) transportation reimbursement for taxi users, and (c) refreshments will be provided. Group session attendance and completion rate (including make-up sessions for absences) data will be collected.

Trained group facilitators (i.e., interventionists) will have experience in group settings, interpersonal skills, and specific KEEP group facilitator training. Group facilitators will be trained over several weeks through a series of phases involving: (a) viewing video records of prior sessions run by experienced KEEP group facilitators, (b) role playing in mock group sessions, with the trainee as a group facilitator, and (c) co-facilitating group sessions with an experienced KEEP group facilitator. Furthermore, KEEP intervention groups are video recorded and supervised by a certified KEEP supervisor and licensed clinician. Supervisors review video records, code each session for fidelity, and provide feedback to the facilitator. Additionally, supervisors lead a weekly consultation meeting for group facilitators.

Services-as-usual

Participants who are randomized into the control condition will have access to standard community-based services. Those services will most likely include individualized therapy for a parent, child, and/or family, psychoeducational programs, and non-KEEP affiliated parent-training programs.

Waves

Since recruitment will be ongoing (i.e. to maximize recruitment) the intervention will need to be delivered in Waves. Participants (of both conditions) will be designated to a particular Wave according to their completion of the baseline assessment and schedule of intervention groups. The first intervention group will start two months after the inaugural baseline assessment is completed, and then a new intervention group will begin every two months. Participants that complete the baseline assessment before the first meeting of a new intervention group will be part of a Wave together. Participants completing the baseline assessment after the start of an intervention group will be designated to the following Wave, and so forth for a total of 12 waves. Only participants in the intervention condition will be notified and eligible to attend the intervention group of their corresponding Wave (i.e., Wave 1 participants in the intervention condition are contacted by the group facilitator of the Wave 1 intervention group, while Wave 1 participants in the services-as-usual condition are not contacted until the final assessment, six months later).

Data collection

Participants will complete a baseline assessment (i.e., pretest) within one month of the child’s new placement and a final assessment (i.e., posttest) six months after the completion of the baseline assessment. Ongoing service utilization will be assessed at the final assessment to control for additional services utilized in both KEEP and services-as-usual conditions.

Measures

Demographics

Participants’ personal identification, such as racial and/or ethnic self-identification, gender, age, foster parent experience, and SES will be collected.

Behavior problems

The Parent Daily Report Checklist (PDR: Chamberlain & Reid, 1987) will be used to assess child behavior problems at baseline and six months later at the final assessment. The PDR is a 30-item measure of child behavior problems. The PDR is administered via telephone to parents on a series of consecutive or closely spaced days (approximately one to three days apart). During each call, a trained RA will ask the primary caregiver the following question: “Thinking about (target child’s name), during the past 24 hours, did any of the following behaviors occur?” RAs will then read the list of 30 behaviors and ask parents to indicate either “yes” or “no” as to whether the behavior had occurred in the last 24 hours. Consistent with previous studies (e.g., Price, Roesch, & Escobar Walsh, 2012), three PDR calls, on different occasions across a two-week period, will be administered at baseline (prior to the intervention) and six months after the baseline assessment. The PDR is structured so that parents focus on recalling only the past 24 hours, thus avoiding aggregate recall or estimates of frequency thought to bias estimates (Stone, Broderick, Kaell, DelesPaul, & Porter, 2000). The PDR has been used in several previous outcome studies, including those with families referred because of child conduct problems (e.g., Kazdin & Wassell, 2000; McClowry, Snow, & Tamis-LeMonda, 2005) and families with children in regular foster care (Chamberlain, Price, Reid et al., 2008; Chamberlain, et al., 1992). The concurrent validity of the PDR has been demonstrated in connection with measures of child and family functioning, including live observations of family interactions in the home (Forgatch & Toobert, 1979; Patterson, 1976) and parents’ ratings of child behavior (i.e., Becker Adjective Checklist; Becker, Madsen, Arnold, & Thomas, 1967). Scores representing levels of child behavior problems will be calculated for each child at baseline and termination by summing the number of behaviors reported per day on the PDR (out of the possible 30) divided by the number of calls made at each assessment period (typically three calls).

Parenting behaviors

The Parenting Young Children (PARYC; McEachern et al., 2012) measure will be used to assess positive parenting behaviors at baseline and final assessments. The PARYC is a brief self-report measure designed to assess the frequency in which parents engaged in three types of parenting behaviors over the past month: (1) Supporting Positive Behavior (e.g., “Notice and praise your child’s good behavior”), (2) Setting Limits (e.g., “Make sure your child followed the rules you set all or most of the time”), and (3) Proactive Parenting (e.g., “Prepare your child for a challenging situation.”). This measure consists of 21-items rated on a 7-point Likert scale with responses ranging from not at all (i.e., 1 pt on Likert scale) to most of the time (i.e., 7 pts on Likert scale) during the last month. Higher scores indicate more positive parenting behaviors. McEachern et al., 2012 found adequate internal consistency and initial validity with the PARYC scales being related to other validated measures of both adaptive and dysfunctional parenting strategies as well as child problem behavior.

Parenting self-efficacy

The Parenting Sense of Competence Scale (PSOC; Gibaund-Wallston & Wanderman, 1978) will be used to assess parenting self-efficacy for both conditions at baseline and final assessments. The PSOC is a 17-item measure of parents’ feelings of self-efficacy in the parenting role and parents’ satisfaction with parenting. Specifically, items are designed to assess a parent’s ability to understand their child’s wants and needs, engage in positive parent-child interactions, and the parent’s personal beliefs regarding their parenting abilities (e.g., “I meet my own personal expectations for expertise in caring for my child”; “Considering how long I’ve been a parent, I feel thoroughly familiar with this role”).  Each item on the PSOC is answered on a six-point Likert scale, with responses ranging from strongly disagree (i.e., 1 pt on Likert scale) to strongly agree (i.e., 6 pts on Likert scale). Higher scores indicate a higher parenting sense of competency and self-efficacy (Gibaund-Wallston & Wanderman, 1978).The PSOC will be administered to primary caregivers at the baseline and final assessments. Internal consistency for the PSOC is in the range of 0.75-0.88, and test-retest reliability ranging from 0.46-0.85 has been reported (Gibaund-Wallston & Wandersman, 1978; Gilmore & Cuskelly, 2008).

Service utilization

We will administer a service utilization questionnaire to all participants at the final assessment. The questionnaire will contain questions regarding hours of medical services, therapy sessions (or other counseling services), psychoeducational groups, and after school programs that parents and children have engaged in throughout the course of their six-month Wave. We will operationalize this variable by compiling the hours across services into a single continuous, quantitative “service utilization” variable. Higher numbers indicate more engagement in other services through the duration of the study.

Planned Analyses

Preliminary

A priori power analyses will be conducted with G*Power 3.1 to determine adequate sample sizes in order for power to be greater than .80 (Cohen’s d = .20), per recommendations by Cohen (1988; Faul, Erdfelder, Buchner, & Lang, 2009). Moderate effect size will be used. All data will be screened for patterns of missingness; casewise deletion will be used. Missing data will be analyzed for systematic patterns in order to reduce bias. Preliminary descriptivestatistics for all variables and covariates will be examined and reported. That process will include additional screening for outliers, missing data, normal distribution, and homoscedacity. We will use statistical modeling to inspect data, ensuring assumptions are met, and then taking corrective actions when necessary. Group differences (between conditions) on baseline variables will be assessed using analysis of variance (ANOVA) and chi-squared tests as appropriate. We will also examine attrition (due to non-engagement, drop out, or change in child placement) rates for differences between conditions. We will examine and report intercorrelations of measures. All analyses will be conducted using IBM SPSS version 23.0.

Main study

Pearson correlation coefficients will be calculated to examine intercorrelations among all variables of interest. Then, we will conduct three separate (i.e., one for each research question) one-way Analysis of Covariance (ANCOVA) tests. The one-way ANCOVA is used to determine whether there are any significant differences between two or more independent (unrelated) groups on a dependent variable. The ANCOVA looks for differences in adjusted means (i.e., adjusted for the covariate). Moreover, the one-way ANCOVA has the additional benefit of “statistically controlling” for a third variable. Nine assumptions must be met before conducting the one-way ANCOVA: (1) dependent variable and covariate variable should be measured on a continuous scale; (2) independent variable should consist of two or more categorical, independent groups; (3) must have independence of observations (i.e., there is no relationship between the observations in each group or between the groups themselves); (4) there should be no significant outliers; (5) residuals should be approximately normally distributed for each category of the independent variable; (6) there needs to be homogeneity of variances; (7) The covariate should be linearly related to the dependent variable at each level of the independent variable; (8) there needs to be homoscedasticity; and (9) There needs to be homogeneity of regression slopes, which means that there is no interaction between the covariate and the independent variable.

We will conduct the first ANCOVA to answer the first research question: “Does receiving the KEEP intervention affect foster caregivers’ use of positive parenting behaviors?” For this test, the categorical independent variable will be treatment condition (i.e., KEEP intervention or services-as-usual) and the continuous, quantitative dependent variable will be parenting behaviors at the final assessment. Parenting behaviors at baseline assessments will be the continuous quantitative covariate (i.e., adding it to the model will control for initial group differences). Results indicating statistically significantly greater use of positive parenting behaviors in the KEEP intervention condition would support our hypothesis.

We will conduct the second ANCOVA to answer the second research question: “Does receiving the KEEP intervention affect parenting self-efficacy?” For this test, the categorical independent variable will be treatment condition (i.e., KEEP intervention or services-as-usual) and the continuous, quantitative dependent variable will be parenting self-efficacy at the final assessment. Parenting self-efficacy at baseline assessments will be the continuous quantitative covariate (i.e., adding it to the model will control for initial group differences). Results indicating statistically significantly higher reported scores of parenting self-efficacy in the KEEP intervention condition would support our hypothesis.

We will conduct the third ANCOVA to answer the third research question: “Will foster caregivers who receive the KEEP intervention report a different amount of toddler externalizing behavior problems than those foster caregivers who do not receive the KEEP intervention?” For this test, the categorical independent variable will be treatment condition (i.e., KEEP intervention or services-as-usual) and the continuous, quantitative dependent variable will be problem behaviors at the final assessment. Problem behaviors at baseline assessments will be the continuous quantitative covariate (i.e., adding it to the model will control for initial group differences). Results indicating statistically significantly fewer toddler behavior problems in the KEEP intervention condition would support our hypothesis.

Discussion

Study Validity

In order to ensure that results from the study would adequately reflect the true relationship between the KEEP intervention, toddler externalizing behavior problems, and parenting self-efficacy, it is necessary to discuss and describe the types of experimental validity and possible threats to validity that exist in the proposed design. Sadish, Cook, and Campbell (2002) designate four types of validity: statistical conclusion validity, internal validity, construct validity, and external validity. In the following paragraphs, each type of validity, along with its potential threats, is reviewed in depth.

Statistical conclusion validity

Statistical conclusion validity refers to the degree to which accurate conclusions about the relationship among variables are made (Heppner, Wampold, & Kivlighan, 2008). Knowing that variables are related involves three questions, which can be addressed by attending to the threats to statistical conclusion validity. The questions are:

(1) Is the study sensitive enough to permit statements about covariation?;

(2) Is there evidence that IV and DV covary?; and

(3) How strongly do IV and DV covary?”

There are seven threats to statistical conclusion validity. First, low statistical power increases the likelihood of making Type II error (i.e., not observing an effect when an there is an actual effect), which threatens the statistical conclusion validity. In this study, we have tried to mitigate the threat of low statistical power by calculating a sufficient sample size using G*Power 3.1 and recruiting accordingly. Nonetheless, there is the possibility that we will not be able to recruit enough participants, in which case, we would increase statistical power by adopting a more lenient alpha level and/or using a one-tailed (versus the standard two-tailed) test when we can support a directional hypothesis with previous literature (Cook & Campbell, 1979). We are also trying to counter the threat of low statistical power by using ANCOVA tests, repeated measures (i.e., pretest and posttest for each participant), and reliable measures (Cook & Campbell, 1979).

Violating assumptions of statistical tests is another threat to statistical conclusion validity because it may increase the probability of making a Type I error (i.e., observing an effect when there is not an actual effect) or Type II error (Cook & Campbell, 1979). We will counter this threat by thoroughly examining each assumption of each statistical test. If assumptions were violated, we would use empirically supported methods to correct the violated assumptions when possible, choose to conduct a different statistical test if supported, and/or do not conduct an analysis and report the results and limitations accordingly.

Similar to violating assumptions of statistical tests, Fishing (i.e., conducting numerous comparisons) is a threat to statistical conclusion validity because it increases the probability of making a Type I error (Cook & Campbell, 1979). We will remove the threat of Fishing by only conducting planned comparisons with properly adjusted alpha levels (i.e., conducting numerous comparisons requires using more conservative alpha levels) (Cook & Campbell, 1979).

The unreliability of measures is a threat to statistical conclusion validity. When a study uses measures that are not reliable, error variance is increased and true effects might not register (i.e., Type II error) or effects could register even though no effect actually occurred (i.e., Type I error) (Cook & Campbell, 1979). In the proposed, we largely control for this threat by using measures that have adequate evidence of reliability in the literature (e.g., PDR, PSOC). Unfortunately, although the PARYC is developed by experienced researchers and shows promise on establishing reliability (McEachern et al., 2012), it is a newer measure that has not been used in enough studies to indicate adequate evidence of reliability, so using it in the study poses a potential threat to statistical conclusion validity. In addition to reporting this potential threat in the limitations sections of the study, we will collect and report our own reliability data (for all measures), which will illustrate the reliability of measures and potentially help establish adequate evidence of reliability for the PARYC.

Administering interventions in a standard fashion is important to statistical conclusion validity because unreliability of treatment implementation (i.e., unstandardized administration of an intervention) increases error variance (Cook & Campbell, 1979). We will attempt to control for this threat by emphasizing standardization in the implementation process, utilizing a manualized curriculum, and monitoring the consistency of delivery (i.e., audio and video recording sessions for observers to track differences and implementation fidelity). Nonetheless, there will likely be variation (e.g., facilitator style) in the implementation of the intervention and that could affect understanding the relationship between variables.

Akin to unreliability of treatment implementation, there could be random irrelevancies in the experimental setting (e.g., a sudden smell of apple pie, hearing sirens, feeling the breeze through a window) that may divert participant’s attention from treatment and possibly wash out true effects of the intervention (Cook & Campbell, 1979). Although we will try to minimize these extraneous factors by using uniform settings (Cook & Campbell, 1979), random irrelevancies in the experimental setting pose a threat to the study. For example, the availability our initial setting may change during the two years, so treatment groups might be administered in alternate settings. Likewise, as a result of having overlapping intervention groups, intervention groups occurring simultaneously will have to take place in a different room, on a different day of the week, or at a different time of day. Those setting differences could introduce error variance.

Random heterogeneity of respondents, which means wide variation among the participants, poses a threat to statistical conclusion validity. Specifically, more variation among participants results in larger error variance and less statistical significance, ultimately, obscuring observation of a true effect (Cook & Campbell, 1979). There is the possibility that the proposed study is susceptible to this threat, but we will attempt to control for it by using repeated measures and conducting an ANCOVA analysis. If we do observe random heterogeneity of respondents, we could match participants on variables correlated with the posttest (Cook & Campbell, 1979).

Internal validity

Internal validity refers to the degree of certainty for inferring a causal relationship between variables by ruling out all the plausible rival hypotheses (Heppner et al., 2008). Moreover, the strength of certainty is assessed by the degree in which alternative explanations for the relationship can be ruled out. In the proposed study, internal validity is comprised of being able to rule out all alternative explanations that could account for a reduction in toddler externalizing behavior problems, so we can conclude that the KEEP intervention reduced toddler externalizing behavior problems.

There are numerous factors that pose a threat to internal validity, so they must be addressed accordingly.

Confounding is a significant threat to internal validity. Confounding refers to the presence of extraneous variables that affect the observed relationship between the independent and dependent variables (Heppner et al., 2008). The current study attempts to minimize the threat of several confounding variables. First, service utilization data will be collected at the final assessment and controlled for in the analyses. The inclusion of service utilization as a control variable lowers the chances that the expected increases in positive parenting skills and parenting self-efficacy are due to engagement in a different counseling service in the community, as opposed to the effect of KEEP alone.

History and maturation effects are threats to internal validity. History effects refer to uncontrollable life events that could occur between the initial and final assessments (Cook & Campbell, 1979). History effects will most likely occur in the study, but the experimental design helps minimizes the effect of history confounds. Also, history effects can be reduced by assessing treatment groups at the same point in time. Maturation refers to normal developmental changes in participants that happen between baseline assessment and final assessment and may affect the results (Heppner et al., 2008). For example, toddler can development rapidly in six months and they might decrease externalizing behaviors as part of natural development. Using a control groups helps mitigate this threat because it is likely to be equal across groups (Heppner, et al., 2008).

Attrition is the effect of participants dropping out of a study, which poses a threat to internal validity in this study. Attrition rates will be will be assessed at the final assessment; it is expected that participants will drop out because maintaining engagement of families with increase risks in intervention studies is challenging (Friars & Mellor, 2007; Thomas & Zimmer-Gembeck, 2011). Further, there is a chance that attrition will not be equal across groups because the intervention group requires a larger commitment, but this variance this variance in attrition will be controlled for in the analytic plan.

Construct validity

Construct validity refers to the degree to which the independent and dependent variables accurately reflect the construct of interest (Heppner et al., 2008). Construct confounding, which occurs when a construct also inadvertently reflects another construct. It is possible that parenting self-efficacy is confounded with other parenting constructs, like parent-empowerment, self-confidence, and sensitivity. We chose to use assessments that have shown to be valid and reliable indicating that the tests measure the intended construct, which reduces the likelihood of construct confounding.

Mono-method bias is another potential threat to construct validity. Mono-method bias occurs when a single method of measurement is used to determine the level of a construct (Heppner et al., 2008). Mono-method poses a threat in this study because all the outcomes are measured by a single method of measurement. Unfortunately, this increases the possibility that confounding constructs and should have been controlled for in the study design.

Experimenter expectancies pose a threat to construct validity. Experimenter expectancies is the conscious or unconscious manipulation of outcomes based on the experimenter, researcher, or interventionist’s anticipated results (Cook & Campbell, 1979). As indicated, researchers are blind to condition, so that will help control for this threat. Although group facilitators are aware of the study and that their group members are participants in the intervention condition of a study, they will not be aware of the research questions or the measures, which will also control for their expectancies.

External validity

External validity refers to the degree to which the causal relationship is generalizable across units, treatments, outcomes, and settings (Cook & Campbell, 1979). The current study draws from a specific population of parent-child dyads within the child welfare system. The children in the dyad tend to be representative of the general population residing in the region, but foster parents tend to be less representative of the general population. Nonetheless, the sample might be quite homogeneous as a result of the region and the highly specific target population. Consequently, findings will likely not be as generalizable to populations outside of the target population and/or region. of the study sample. Similarly, a threat to external validity exists in regard to how well the causal relationship holds up across other settings. There are often differences among treatment settings, thus results obtained could be different if KEEP were delivered in a school versus a church. Thus, the causal outcome produced by the current study may not be fully generalizable across treatment settings.

Another threat to external validity relates to treatment variations (i.e., how the treatment varies). Although KEEP is a manualized intervention, and every effort will be made to provide a standard protocol of KEEP to study families, there might be differences between the delivery of KEEP in the context of a research study as compared to KEEP delivered in the community. The generalizability of the results across variations of treatment is somewhat threatened. Nevertheless, variation while following manualized treatment, poses less threat than using unstandardized services.

Limitations

There are limitations to the proposed study that need to be acknowledged and addressed.  First, KEEP is an intensive intervention that requires parents to meet for two hours every week as well as practice strategies at home. While KEEP is manualized and integrity of implementation will be maintained, it is impossible to ensure that all families will engage as expected. Therefore, treatment integrity is not in the experimenter’s control. This would impact efficacy of KEEP, and would have a considerable effect on the outcomes of interest. Another limitation to this study is the recruitment process and the intervention timeline. The intervention needs to be administered on a schedule to mitigate internal validity threats, but children do not enter the child welfare system on a schedule. Consequently, some groups may have many participants, and some groups could have few or none. Similarly, implementing 12 waves of parent training groups will require vast amount of human and physical resources, and it will be a challenging feat logistically, especially to implement the intervention with uniformity.

Strengths

Although there are limitations to this study, there are also several strengths. First, proposing a study that uses a longitudinal pretest/posttest, randomized control experimental design is a strength. The longitudinal design allows us to establish a sequence of events (i.e., observing before and after data) and provides the opportunity to observe change over time (Heppner et al., 2008). Moreover, we have increased confidence in concluding causal relationships between predictor variables and the outcome variables of interest by using a longitudinal design (Heppner et al., 2008). Random assignment is another strength of the proposed study. Randomly assigning participants to the treatment condition (i.e., KEEP intervention or services-as-usual) increases the chances that participant characteristics will be equally represented across conditions, and that helps restrict error variance resulting from group composition (Heppner et al., 2008). Additionally, using random assignment affords the most accurate analysis of the intervention effect because the impact of KEEP on toddler externalizing behavior problems is evident regardless of confounding factors that may also impact participants. Lastly, the potential contribution to current research on the KEEP intervention mechanisms of change, along with research on parenting self-efficacy, is a strength of this study. Moreover, this study parallels previous studies and would increase the generalizability of KEEP efficacy to varying populations in different environments.

Implications

The proposed study would be an important contribution to research demonstrating the effectiveness of KEEP in reducing behavior problems of toddlers in foster care. By reducing their behavior problems, they are less likely to experience placement disruptions and will have a better chance of achieving permanency. Ultimately, toddlers would be staying in foster care for less time, and there would be fewer toddlers in the child welfare system. Additionally, the proposed study would contribute to literature supporting parenting self-efficacy as an important change mechanism. Specifically, the results would demonstrate that the KEEP intervention affects change through parenting self-efficacy, and parent’s beliefs about their own parenting ability influences young child outcomes. Therefor, it might be important to further understand parenting self-efficacy and to include parenting self-efficacy in the development of future parenting interventions With the prior in mind, future research could explore how parenting self-efficacy is cultivated, how it functions over time (e.g., “does parenting self-efficacy remain constant over time?”), and how it can be incorporated into parenting interventions as a primary target.

References

Branje, S. J., Hale, W. W., III, Frijns, T., & Meeus, W. H. (2010). Longitudinal associations between perceived parent-child relationship quality and depressive symptoms in adolescence. Journal of Abnormal Child Psychology, 38(6), 751–763.

Cappa, K. A., Begle, A. M., Conger, J. C., Dumas, J. E., & Conger, A. J. (2011). Bidirectional relationships between parenting stress and child coping competence: Findings from the PACE study. Journal of Child and Family Studies, 20, 334–342.

Chamberlain, P., Price, J. M., Reid, J. B., Landsverk, J., Fisher, P. A., & Stoolmiller, M. (2006). Who disrupts from placement in foster and kinship care? Child Abuse and Neglect, 30, 409-424.

Child Welfare Information Gateway. (2016). Foster care statistics 2014. Washington, DC: U.S. Department of Health and Human Services, Children’s Bureau.

Cook, R. J. (1994). Are we helping foster care youth prepare for their future? Children and Youth Services Review, 16, 213-229

Crawford, A., & Manassis, K. (2001). Familial predictors of treatment outcome in childhood anxiety disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1182–1189.

Creasey, G., & Reese, M. (1996). Mothers’ and fathers’ perceptions of parenting hassles: Associations with psychological symptoms, nonparenting hassles, and child behaviour problems. Journal of Applied Developmental Psychology, 17, 393–406.

Crnic, K., & Greenberg, M. (1987). Maternal stress, social support, and coping: Influences on early mother–child relationship. In Research on Support for Parents and Infants in the Postnatal Period, Boukydis C (ed.). Ablex: Norwood, NJ; 25–40.

Deater-Deckard, K. (1998). Parenting stress and child adjustment: Some old hypotheses and new questions. Clinical Psychology: Science and Practice, 5, 314–332.

Deater-Deckard, K., & Scarr, S. (1996). Parenting stress among dual-earner mothers and fathers: Are there gender differences? Journal of Family Psychology, 10, 45–59.

Dix, T. (1991). The affective organization of parenting: Adaptive and maladaptive processes. Psychological Bulletin, 110, 3–25. doi:10.1037/0033-2909.110.1.3

Dodge, K., Pettit, G., & Bates, J. (1994). Socialization mediators of the relation between socioeconomic status and child conduct problems. Child Development, 65, 649–665.

Festinger, T. (1983). No One Ever Asked Us: A Postscript to Foster Care. New York: Columbia University Press.

Greenberg, M.T., & Lippold, M.A. (2013). Promoting healthy outcomes among youth with multiple risks: Innovative approaches. Annual Review Public Health, 34, 253-270.

Halgunseth, L.C., Perkins, D. F., Lippold, M.A., & Nix, R.L. (2013). Delinquent-oriented attitudes mediate the relation between parental inconsistent discipline and early adolescent behavior. Journal of Family Psychology, 27(2), 293-302.

Jackson, A. (2000). Maternal self-efficacy and children’s influence on stress and parenting among single black mothers in poverty. Journal of Family Issues, 21, 3–16.

Karrass, J., VanDeventer, M., & Braungart-Riker, J. (2003). Predicting shared parent–child book reading in infancy. Journal of Family Psychology, 17, 134–146.

Kim, S., Kochanska, G., Boldt, L. J., Koenig Nordling, J., & O’Bleness, J. J. (2014). Developmental trajectory from early responses to transgressions to future antisocial behavior: Evidence for the role of the parent-child relationship from two longitudinal studies. Development and Psychopathology, 26, 93-109.

Kufeldt, K., Armstong, J., & Dorosh, M. (1989). In care, in contact? In J. U. Hudson & B. Balaway (Eds.), The state as parent (pp. 355-368). Dordrecht, The Netherlands: Kluwer Academic.

McKay, J.S. (2003). Caretaker bias in the study of young children. Journal of the American Academy of Child and Adolescent Psychiatry, 42(11), 1267. DOI: 10.1097/01.chi.0000087566.61396.5d

Millham, S., Bullock, R., Hosie, K., & Haak, M. (1986). Lost in care, Gower, London.

Moffitt, T. E., & Caspi, A. (2007). Evidence from behavioral genetics for environmental contributions to antisocial conduct. In J. Grusec & P. Hastings (Eds.), Handbook of socialization (pp. 96–123). New York, NY: Guilford Press.

Newton, R.R., Litrownik, A.J., & Landsverk, J.A. (2000). Children and youth in foster care: Distangling the relationship between problem behaviors and number of placements. Child Abuse and Neglect, 24(10), 1363-1374.

Oregon Department of Human Services: Children, Adults and Families Division. (2016). 2015 Child Welfare Data Book.

Pahl, K. M., Barrett, P. M., & Gullo, M. J. (2012). Examining potential risk factors for anxiety in early childhood. Journal of Anxiety Disorders, 26(2), 311-320

Pardeck, J. T. (1984). Multiple placement of children in foster family care: An empirical analysis. Social Work, 29, 506-509.

Patterson, G. R. (1982). Coercive family process. Eugene, OR: Castalia.

Penzerro, R., & Lein, L. (1995). Burning their bridges: Disordered attachment and foster care discharge. Child Welfare, 74(2), 351-367.

Pinderhughes, E. E., Dodge, K. A., Bates, J. A., Pettit, G. S., & Zelli, A. (2000). Discipline responses: Influence of parents’ socioeconomic status, ethnicity, beliefs about parenting, stress, and cognitive-emotional processes. Journal of Family Psychology, 14, 380–400.

Rodriguez, C., & Green, A. (1996). Parenting stress and anger expression as predictors of child abuse potential. Child Abuse and Neglect, 21, 367–377.

Rubin, K., Stewart, S., & Chen, X. (1995). Parents of aggressive and withdrawn children. In Handbook of Parenting: Status and Social Conditions of Parenting, Borstein M (ed.). Erlbaum: Mahwah, NJ; 255–277.

Rubin, D.M., O’Reilly, A.L., Luan, X., & Localio, A.R. (2007). The impact of placement stability on behavioral well-being for children in foster care. Pediatrics, 119(2), 336-344.

Sedlak, A., & Broadhurst, D. (1996). The third national incidence study on child abuse and neglect (NIS-3). Washington, DC: U.S. Department of Health and Human Services.

Siever, L. J. (2008). Neurobiology of aggression and violence. The American Journal of Psychiatry, 165, 429–442. doi:10.1176/appi.ajp.2008 .07111774

U.S. Census Bureau (2010). QuickFacts. Retrieved from https://www.census.gov/quickfacts/fact/table/portlandcityoregon/PST045216

U.S. Department of Health and Human Services, Administration for Children and Families, Ad­ministration on Children, Youth and Families, Children’s Bureau. (2013). Child Maltreatment 2012. Retrieved from http://www.acf.hhs.gov/programs/cb/research-data-technology/statistics-research/child-maltreatment