Total Client Sampling and a Test of Family Preservation Services
 
 
 
 
 
 

Kim A. Lesters, D.S.W.
POB 71317
Las Vegas, Nevada 89170-1317

 
 
 
 
 
 
 
 
 


 Total Client Sampling and a Test of Family Preservation Services
 
 

 Abstract

 This study evaluates a five year quasi-experiment in the provision of family preservation to families at imminent risk of placement, comparing outcomes with a five year baseline experience and "customary services" program in a similar city.  Clients in the experimental group received concrete services and a large amount of psychotherapeutic care of one sort or another.  Clients in the two comparison groups probably received similar amounts of concrete services but much less therapeutic care.  Clients in the experimental group were placed at a much lower rate than clients in either comparison group.  Moreover, the home environment of experimental clients also improved much more than the home environment of controls.

 Total Client Sampling and a Test of Family Preservation Services
 A considerable controversy has emerged over the provision of family preservation services particularly because of contradictory findings in a number of state experiments, demonstrations, and research evaluations.  While usually modest, one group of studies has consistently found positive results from family preservation interventions (Kinney et al. (1977); Kinney et al. (1991);  Thiemann, Fuqua, and Linnan (1990); Wheeler et al. (1993); Berry (1992);  Bergquist, Szwejda, and Pope (1993).  However, many of these studies have been faulted for not employing rigorous research designs (Schuerman, Rzepnicki, and Littel 1994;  Rossi 1991).  Another group of studies, employing more rigorous designs have found little positive effects for family preservation (Schuerman, Rzepnicki, and Littel 1994; Walter R. McDonald and Associates (1990); Feldman, 1990, 1991).  Nevertheless, a number of studies that did use randomized designs (Jones 1985); Mitchell, Tovar, and Knitzer (1989); Pecora, Fraser, and Haapala 1992 ) found that family preservation was somewhat successful;  however, they too were criticized for their many methodological failures.  Still, McCroskey and Meezan (1993) reported that family preservation services actually increased placements.
 Family preservation services have been adopted as a major child welfare strategy and incorporated into policy through PL 96-272.  Many of the programs have been modeled after the Homebuilders program in Washington state and the considerable variation among contemporary family preservation strategies still contains a number of common elements at its core.  Yet it is commonly agreed that placement should be still undertaken where there is an obvious danger to the physical well being of the child or any other member of the family.  Family preservation is intended for families with a child "at risk of imminent placement."  Family preservation services should start quickly;  they should be family centered;  there should be a moratorium on placement during services (except when there is an obvious threat of physical danger);  and clients should be provided intensive services, often involving visits of serval hours duration more than once a week.  Moreover, most family preservation interventions funded through Federal provisions in the US also include a number of other elements: continuity of care, low caseloads, emergency financial aid, and "suitability" (families in which there is at least one cooperative adult.
 A number of methodological criteria for evaluating family preservation services have emerged through the past few years of research efforts that have been reported in an expanding and increasingly sophisticated literature.  First, samples should be large;  second, comparisons should be made with appropriate control groups; and third, follow-up data need to be collected.  Finally, many of the studies suffer from one form of selectivity bias or another:  either the study samples were compromised by selecting populations for study that were unrepresentative of the underlying population of concern (children at imminent risk of placement) or a form of bias intruded into the randomization procedures themselves.  Thus, studies need to institute a variety of procedures to assure sampling representativeness and measurement accuracy.
 The present study adds to the growing literature on the outcomes of family preservation.  It offers a number of design innovations in creating a relatively credible evaluation of family preservation in a large Southwestern city (the City) that is representative of the general American experience.
Method
 The State instituted a renovated family preservation program (RFP) in 1996, funneling money through the City for the provision of all publicly funded family preservation services within its boundary.  In turn, the City contracted out or delegated all of its RFP care to the large, long-standing social service agencies that were already receiving funds for public social services.  When the agencies could not provide the service themselves, they subcontracted with specialized agencies.  Five of these seven contract agencies had been operating for more than 25 years each and two for more than 15 years;  all employed social workers and social work interventions at their core.  They were highly professionalized organizations.  Four of the agencies had religious affiliations;  one was a traditional "child and family" style counseling agency; and two were city agencies (the City's own Division of Family and Child Services and the community mental health complex in the municipal hospital).  From the outset, the decision was made to systematically evaluate the experience.  The author was contracted to design and implement that evaluation.
 RFP is evaluated through its five years of operation by comparing its rates of placement with rates of placement in a family preservation program in a similar comparison city (CC) (also in the state) and to a baseline period of five years (1991 through 1995) before RFP was instituted.  In addition, a series of before/after comparisons along a number of other dimensions were conducted for RFP clients.
 RFP.   In planning for RFP, all of the seven agencies agreed to provide a common core of services, largely modeled after the Homebuilders approach.  The focus of the services was on improving family functioning through intensive case work while removing barriers to other services -- employment, counseling, training, education, and so forth -- that would enhance the ability of the client families to function at least adequately to prevent placement.  Moreover, the caseworkers felt that their services were constructed around a multisystems approach to family functioning (Aponte and Van Deusen 1981; Minuchin and Fishman 1981).  This entailed small caseloads, quick initial responses, follow-up, and time-limited interventions.
           Upon referral to the child protection agency, each case (family) was interviewed by a caseworker with an MSW degree who assessed risk of placement (see below).  When the family was judged to be at imminent risk of placement, the family was assigned to a family preservation caseworker, also with an MSW, who took over treatment planning and referrals for concrete services.  Relative to baseline services (Table 1) nearly all of the large differences in therapeutic care are significant while only two of the concrete services differences are significant and these only at the .05 level.
 RFP clients received a large amount and considerable intensity of therapeutic services, even more than reported by Homebuilders or Putting Families First.  In reference to Table 1 it is quite apparent that RFP clients received about five times more therapeutic services than clients during the baseline period.  Indeed, it is obvious that any differences that emerge in outcomes, notably placement rates, can be fairly attributed back to effects of therapeutic services.  Especially prominent are the large proportions of the RFP caseload that received family counseling (80.2%), individual counseling (55.3%), and treatment for alcohol and drug abuse (65.3%).  Relatively little of this care was provided to baseline clients.  In the end, RFP clients received on average more than seventy hours of therapeutic care above the small amounts received by baseline clients.  Again, on average, RFP clients received therapeutic care over more than fourteen months while baseline clients averaged only 4.7 months.  It is also worth noting that while both groups received similar types and intensities of concrete services, RFP clients received them over a much longer period of time (11.4 vs. 5.2 months).
 Samples and Comparison Groups.  State law prevented denying any family these services thus blocking the use of a randomized procedure for assigning clients.  Therefore, the evaluation adopted a design that included all RFP families in the analysis from three months after the program began (January 1 1996) for five full years (ending on December 31, 2000).  Total client sampling greatly reduced at least one form of selectivity bias since all cases were included.  Nevertheless, the referrals for family preservation may have been biased for amenable cases.  Without randomized controls the extend of "creaming" cannot be estimated.
 Fortunately, the State also contains another large city (the comparison city, CC), very similar to this one.  Indeed, in population, economic characteristics, ethnography, and need, the two cities are nearly identical.  The CC did not incorporate a family preservation program until 1997 and in the event a relatively restricted one.  Therefore, the design included a natural comparison group (CC).
 In addition, RFP was compared with care given to clients during the five years prior to RFP (1991-5 inclusive).  The baseline program was an underfunded family preservation prevention program that was similar to many of the "customary service" programs that the literature compares with family preservation efforts.  Data to describe the baseline services were drawn from the case records.
 Services for clients in the comparison city were less systematically described although it is generally believed that they received baseline care since the per capita costs of both baseline and CC care were similar ($1652 and $1441 respectively).  In contrast total per capita costs for RFP clients averaged $4726 with the differences roughly proportionate to the differences in the intensity of total services (notably attributed to therapeutic care). Unfortunately imminent risk was not measured in either the baseline or CC, the two comparison groups; clients there were simply referred for additional services at the discretion of their caseworkers and child welfare investigators.
 Table 2 suggests that in spite of the absence of any randomization procedure, the families in all three groups were roughly equivalent. Indeed, the few statistically significant differences (% two parent families, % minority, and % with prior referrals) were not substantial.
 Data collection.  Data was collected from case records, interviews with clients, and home visits.  Placement decisions were gathered from the case record.  The quality of the home environment was measured by scores on the Family Environment Scale (FES) (Moos and Moos 1994) that was administered by the caseworker, first during home visits when the family was referred and again when the case was terminated (either because of successful preservation or six months after the children's placement).  FES measured the quality of the foster home, adoptive home or institution relative to the initial home environment.  Satisfaction was measured by caseworker interviews with the children and parents.  The caseworker's satisfaction was measured through interview with the researcher or a research staff assistant.  Satisfaction was assessed through the Frankenhess Satisfaction Scale (Lowell and Mein 1992).  Caseworkers were only interviewed for their satisfaction if they had had responsibility for the case for more than six months.  The only outcome measures possible in the comparison groups were placement rates, obtained by record review.
 Placement rates were computed for RFP clients by a review of all records on the closing date of the experiment: December 31, 2000.  Baseline data were collected from records on December 31, 1995 and for the comparison city on December 31, 2000 going back four years.
 Imminent risk.  Determination of imminent risk by the child welfare intake worker was the deciding factor in referral for RFP.  Imminent risk was determined on the basis of a seventeen point questionnaire administered by the in-take worker during the first interview that assessed child and family status and functioning.  The adapted questionnaire was modeled on the work of Praegle and Rosensweig (1994).  The adapted instrument's sociometric properties as well as the agency planning process to implement family preservation services is reported separately.  With little variation between years, only an average of 14.3% of referred families refused services.
 For each year of their operation -- the City after RFP, the City for baseline years, and CC -- Table 3 presents the annual child welfare caseload and the annual referrals to family preservation.  As can be seen all groups ran their family preservation efforts at roughly twenty percent of their child welfare caseloads.  Interyear differences were tested against data adjusted by population growth with the result that no significant differences emerged.  Furthermore, separate analyses revealed that no agency experienced any significant variation in its inter-year pattern of referrals.
 In the end then, this study evaluates a five year quasi-experiment in the provision of family preservation to families at imminent risk of placement, comparing outcomes with a five year baseline experience and "customary services" program in a similar city.  The findings provide valuable estimates of RFP's outcomes that allow for conclusions that are nationally relevant.
Findings
 Table 4 reports that out-of-home placement rates over five years were significantly and substantially lower for RFP clients than during the baseline or in the CC.  Only 27.1% of RFP clients, but 48.9% of baseline clients and 51.3% of CC clients, were placed during the study years of the experiment.  Moreover, while placement rates increased slightly over the five years, the few statistically significant increases were not substantial (Table 5).  Indeed RFP placement rates increased much less over time than in comparison with both baseline and CC clients.
 The environment of the RFP children substantially and significantly improved over the course of the RFP services by 19 points to reach 37 on the FES scale, passing the point (32) of a generally acceptable home environment for children (Table 4).  It is notable that the environments of placed children improved from 14 to 35 on the FES scale.  It is important to note too that the FES scale includes many items on psychological adjustment and comfort.  Consistent with these findings, children, parents and caseworkers reported considerable satisfaction with the services at the time of termination.
Conclusions and Discussion
 The present study evaluated the ability of an intensive family preservation program to prevent family dissolution.  The particular conditions of RFP -- intensive therapeutic services together with many concrete services -- in comparison with two controls that lacked any intensive core of therapeutic care suggest a number of conclusions about family preservation services.  The basic conclusion is that intensive family preservation that includes a large core of therapeutic care is superior to concrete services alone in maintaining families at imminent risk of placement.  In particular, the home environment of families that went through preservation services substantially improved.  It is also noteworthy that services were delivered in a manner that satisfied both parents and children.  Workers also were satisfied with the manner of care.
 While the evaluation could not be conducted through randomized controlled procedures, the particular conditions of the present experiment enhance the credibility of conclusions.  First, the samples in all three groups were large and comparable; the services were provided for a substantial number of years (at least four); multiple measures of outcomes were taken; placement rates, the principal outcome measures, were relatively hard and definitive;  and the data appear to be reliable and accurate.
 These conclusions and the credibility of research tend to refute pessimism, notably Rossi's (1991) and Schuerman, Rzepnicki, and Littell's (1994) but also including others, toward family preservation care and the ability of professional social work to routinely assist troubled families.  It is quite apparent, that clients in RFP fared better than similar families receiving only concrete care.  In the end, the findings reinvigorate trust in the psychological nature of many family problems, suggesting that after a minimal amount of concrete care is provided the principal barriers to successful family life may be motivational, cognitive, and attitudinal.  Short-term, intensive, therapeutically grounded attention to families at imminent risk of dissolution is probably an effective helping strategy of contemporary social work.
 
 
 


References
 Aponte, H. and J. Van Deusen. 1981. "Structural Family Therapy." p.310-360 in Handbook of Family Therapy, edited by A. S. Gurman and D. P. Kniskern.  New York: Brunner/Mazel.
 Berquist, C., D. Szwejda, and G. Pope. 1993. Evaluation of Michigan's Families First Program: Summary Report. Lansing: Mcihigan Department of Social Services.
 Berry, M. 1992. "An Evaluation of Family Presrvation Sevices: fitting Agency Services to Family Needs." Social Work 37(4):314-321.
 Feldman, L. H. 1990. Evaluating the Impact of Family Preservation Services in New Jersey. Trenton: New Jersey Division of Youth and Family Services; Buyreau of Research, Evaluation, and Quality Assurance.
 Feldman, L. H. 1991. Assessing the Effectiveness of Family Preservation Services in New Jersey within an Ecological Context. Trenton, New Jersey: Division of Youth and Family Services; Buyreau of Research, Evaluation, and Quality Assurance.
 Jones, M. A. 1985. A Second Chance for Families, Five Years Later: Follow-up of a Program to Prevent Foster Care.  New York: Child Welfare League of America.
 Kinney, J. M., B. Madsen, T. Fleming, and D. A. Haapla. 1977. "Homebuilders:Keeping Families Together," Journal of Consulting and Clinical Psychology 43(4):667-73.
 Kinney, J. M., D. Haapala, and C. Booth. 1991. Keeping Families Together: The Homebuilders Model Hawthorne, NY: Aldine de Gruyter.
 Lowell, S. and R. L. Mein. 1992.  "Psychometric Properties of the Frankenhess Satisfaction Scale" Journal of Sociometry and Psychometrics 15(3):271-279.
 McCroskey, J. and W. Meezan. 1993. Outcomes of Hom Based Services: Effects on Family functioning, Child Behavior and Child Placement. Unpublished paper.  Los Angeles: University of Southern California School of Social Work.
 McDonald, W. R. and Associates. 1990.  County Family Preservation Programs Evaluation: Analysis of Similar Family Preservation Program Evaluations.  Paper submitted to the Office of Child Abuse Presevention, California Department of Social Services. Sacramento CA: Author.
 Minuchin, S. and C. Fishman. 1981. Family Therapy Techniques. Cambridge, MA: Harvard University Press.
 Mitchell, C. P. Tovar, and J. Knitzer. 1989. The Bronx Homebuilders Program: An Evaluation of the First 45 Families. New York: Bank Street College of Education.
 Moos, R. H. and B. S. Moos. 1994. Manual: Family Environment Scale Palo Alto, California: Consulting Psychologists Press.
 Pecora, P. J., M. Fraser, and D. A. Haapala. 1992. "Intensive Home-Based Family Preservation Services: An Update from the FIT Project." Child Welfare 71(2):177-88.
 Praegle and Rosenzweig. 1994.
 Rossi, P. H. 1991.  Evaluating Family Preservation Programs: Report to the Edna McConnell Clark Foundation. Amherst: University of Massachusetts Social and Demographic Research Institute.
 Schuerman, J. R., T. L. Rzepnicki, and J. H. Littell. 1994. Putting Families First.New York: Aldine de Gruyter.
 Thieman, A. A., R. fugua, and K. Linnan. 1990. "Iowa Family Preservation Three Year Pilot Project: Final Evaluation Report." Ames: Iowa State University.
 Wheeler, C. E., G. Reuter, D. Struckman-Johnson, and Y. T. Yuan. 1993. "Evaluation of State of Connecticut Intensive Family Preservation Sevices: Phase V Annual Report." Sacramento, CA: Walter R. McDonald and Associates.