Whatever
your political views, you likely share in concerns many hold over the
difficulties facing socially and economically disadvantaged families in the
U.S. But can the government do anything to directly help such families through family-strengthening
efforts? Despite all you might have heard to date, there is some good news
emerging from recent studies and my goal here is to describe that news.
The
U.S. Administration for Children and Families, specifically the Office of
Family Assistance (OFA), is currently invested in three specific components of family
strengthening, including efforts to a) improve the quality and stability of the
relationships of couples with children, b) increase father involvement for
those with fragile relationships with their children, and c) increase the
quality of co-parenting between adults who have children in common but who are
no longer in an ongoing relationship. My focus here is on research on the first
of these types of efforts, as there has been considerable attention to the
results of two major federal evaluations of programs aimed to help couples with
low incomes and other disadvantages.
The
large, multi-site studies were called Building Strong Families (BSF) and
Supporting Healthy Marriage (SHM). I believe these studies received so much attention,
in part, because they were connected with the somewhat controversial,
government-funded efforts over the last twelve years to provide relationship
education in support of the goals in the welfare reform law of 1996 to promote
two-parent families. These two studies produced disappointing results, which
have been lamented among those who support such efforts and trumpeted by those
who are critical of such efforts. I think the trumpets have mostly carried the
day.
But
there is some good news for
supporters of relationship education in recent findings, including within those
two federal studies. Specifically, it is becoming clear that ethnic minority
couples benefit at least as much or more than other couples from such programs.
Some have suggested that this would not be the case because such programs were
originally designed and tested with mostly middle-class, white couples.[i]
Further, there is newly published evidence that, within the BSF study data set,
the participants who were the most socially and economically disadvantaged
benefitted the most in terms of impacts on relationship quality.
Before
describing in more detail what I see as relatively encouraging, I will first
describe a bit of background on these important federal studies and their
findings to provide context for those who know little about them.
Building
Strong Families (BSF) was a study of unmarried couples in the transition to
parenthood, whereas Supporting Healthy Marriage (SHM) was a study of married
couples. Both projects focused on couples at lower incomes, generally under 200
percent of the poverty line. In both cases, couples in multiple cities were
randomly assigned to either receive or not receive a substantial program of
relationship education and other couple support services, all designed to
strengthen these families with regard to couples’ relationship quality, child
outcomes (including father involvement), and stability. While there are some
disagreements about the results, it is fair to say that the overall evidence
suggested largely non-significant (BSF) or small (SHM) program impacts.
As
a way to provide a bit more detail, allow me to give two brief summaries of the
findings, as if from the perspectives of people who were either more or less
encouraged by the results.
Less
encouraged. The overall findings for BSF showed no evidence of positive
impacts on couples’ relationships and father involvement. There were even some
modest negative impacts in some sites. While there was a modest positive impact
on child behavior, there were no other overall positive effects on child
wellbeing. For SHM, while there was a range of statistically significant
positive impacts, the impacts were modest. Further, these particular programs
were expensive—much more so than most all historical efforts involving
relationship education. In both cases, there was precious little evidence of
positive impacts related to child outcomes. There were some very tiny positive
impacts for children in the SHM study, but tiny is tiny.
More
encouraged. In the BSF study, one site (Oklahoma) did a particularly
impressive job of getting couples in and through services—far outperforming
other sites in this respect. The other sites did not, for the most part, get a
lot of couples through much of the planned services. Across the whole study,
only 55 percent of couples attended any of the relationship education services.
Only Oklahoma demonstrated a range of significant and positive impacts on couple
relationship outcomes at the 15-month assessment.[ii]
While these impacts faded at the 36-month assessment, the children born to
couples in the program group were 20% more likely than children born to couples
in the control group to have lived continuously with both parents until that
3-year point—also, only in the Oklahoma site.[iii]
In SHM, couples showed statistically significant gains at the 12-month
assessment and these gains, while small, were largely maintained out to the 30-month
assessment. In a field where most policy evaluations of social programs show no
significant, lasting impacts, some see this as promising even as the need for
improvement is obvious.[iv]
That’s
the skinny version of what happened. There are detailed reports and endless
commentary on the internet, if you want more information.[v]
The dominant story across the media about these studies is that nothing worked.
However, there was some good news in how SHM sites learned from experiences in
the BSF study, and thus achieved far greater participation and follow-through
among couples than BSF sites did. That is, by including strategies to reduce
barriers to participation and reinforce attendance, SHM enabled more disadvantaged
couples to attend a substantial amount of program services. That is
encouraging.
Before
moving on, I should mention that there is a whole world of research on relationship
education that I am not attempting to cover here, with studies showing
generally stronger, positive effects. Further, some experts in that field have
been dismayed that so much attention has been focused on BSF and SHM in recent
years. I will, however, retain a similarly narrow focus in order to cover findings
related to social and economic disadvantage.
The Response of the Most
Disadvantaged Couples
To
my knowledge, with one exception, the only analyses done to date on the BSF
data set have been conducted by the professional evaluation team hired by the
Administration for Children and Families to conduct the study. The exception
lies in analyses conducted by Paul Amato at Penn State. Amato has approved
access to the BSF data set out to the 15-month assessment point, and he has just published a paper with important analyses from that data set.[vi]
Amato
sought to assess whether couples at greater disadvantage received more,
similar, or less benefits in BSF than other, less disadvantaged couples did. He
analyzed outcomes related to both relationship stability (whether couples broke
up) and relationship quality. His method,
which was different from any other analysis I have seen to date in this field, was
to create a “disadvantage index” based on eleven factors in order to assess
whether having a high or low score on this index affected how much couples
benefited from the programs. I will quote from Amato’s paper regarding the list
of factors going into this index (p. 347):
(a) the mother was less
than 20 years old,
(b) the father was less
than 20 years old,
(c) the mother did not
have a high school degree,
(d) the father did not
have a high school degree,
(e) the father was
unemployed at baseline,
(f) the father earned
less than $10,000 in the last year,
(g) the mother received
public assistance in the last year (TANF, food stamps, Medicaid, SCHIP, SSI,
SSDI, or WIC),
(h) the mother had one
or more children from a previous relationship,
(i) the father had one
or more children from a previous relationship,
(j) the mother or father
reported no one to care for the baby in an emergency (excluding the partner),
(k) the mother or father
reported no one to borrow money from in an emergency (excluding the partner). I
[Amato] omitted mothers’ income from the index because the majority of mothers
were not in the labor force.
While
I need to skip over some technical detail here, I want to note that Amato
approached the analyses in a particularly robust way. He tested his findings
for what we call sensitivity to various specifications. Essentially, he tried
the analyses with various sites left in or out and with various indicators of
disadvantage in or out. The findings were robust across such tests.
Before
testing for the program impacts, Amato found what one would expect: “The
relationship risk variable revealed that higher disadvantage scores were
associated with less support and affection, more destructive conflict, less
constructive conflict, less trust in partner, more intimate violence, and lower
overall relationship quality” (p. 350). Thus, the risk index captures risk as it
was designed to do. The important question is, did their level of disadvantage
matter for how much benefit couples received in the BSF intervention model? While
disadvantage level did not matter for how the program affected couples’ stability
(their odds of breaking up), when it came to relationship quality, those with
more disadvantage received the most positive impact from the programs.
Quoting
from Paul Amato’s paper:
One of the major
criticisms of BSF programs for unmarried couples (and federally funded marriage
education programs for low-income couples) is that educational interventions
are not effective for disadvantaged populations (Johnson, 2012; Karney &
Bradbury, 2005). It is reasonable to imagine that poor couples are so
overwhelmed by financial problems and everyday stress that they are
unresponsive to relationship education programs and see them as largely
irrelevant to their lives. If this were the case, then the most disadvantaged
couples – those most at risk of relationship problems – would receive the least
benefit from programs like BSF. This study, however, suggests the opposite:
Contrary to the notion that disadvantaged couples do not benefit from
relationship education, these couples may be the main beneficiaries of these
services, provided that they are able to keep their unions intact. (p. 353)
Keep in mind, these results are for the 15 month follow-up. It is possible that if the same analyses are one day repeated for the 36 month
follow-up, this same result would not be found. It is not unusual in this field
to find impacts in an earlier period that fade by the time a later follow-up is
conducted.
In
contrast to these encouraging findings from Amato’s paper, results from a meta-analysis
working its way toward publication suggest that the very poorest couples
receive the least benefit from such programs.
(I have the author’s permission to mention what I know about the
analyses.) I believe, however, that the type
of analysis in this other study is far less sensitive to addressing the question
Amato tested. Nevertheless, the findings from this other study align more
closely with the arguments made by the researchers noted in Amato’s quote above,
who have suggested that severe economic hardship may interfere with couples’
ability to benefit from such efforts. It is not hard to imagine that chaos and
stress would interfere with learning new strategies in one’s relationship. On
the other hand, when studies in this field do analyze whether impacts vary based
on levels of prior risk, those at greater risk often get the most benefit. There
is a lot of complexity here for researchers in the field to sort out.
Amato’s
analyses are serious and thoughtful, and he obtained a potentially important
finding that is not at all evident from the primary analyses conducted with the
BSF data set. That takes nothing away from the main results in BSF (pooled
across sites) that are legitimately disappointing for reasons about which serious
people may not agree. But Amato’s analyses are encouraging, and perhaps even
provocative, for suggesting that such services may actually provide the most
benefit, on average, to couples with some of the greatest disadvantages in life.
In fact, Amato goes so far as to imply that if the BSF study had recruited substantially
more disadvantaged couples, the overall findings across the study would have
been positive (p. 353).
Amato’s
findings are not unprecedented. They are the most sophisticated version of a
type of finding that has been obtained before, wherein those who are more
disadvantaged receive at least as much, and sometimes more, benefit from
relationship education services than others.[vii]
Amato notes that this is generally the case for various social programs (p.
353). What he found is also consistent with other studies focused on family
strengthening that find positive impacts for programs given to highly disadvantaged
couples and families. For example, Phil and Carolyn Cowan and their colleagues
have demonstrated positive impacts from a program focused on father involvement
in a study with low-income families, with a particularly large representation
of Mexican American families (67 percent of participants). They found significant,
positive impacts on couple relationship quality, father engagement, and children’s
problem behaviors.[viii]
The Response of Ethnic Minority
Couples
On
to other encouraging news I want to share. In both the BSF and SHM studies, the
evaluators were able to examine if the minority group with the largest
representation got more or less impact than other couples. The largest minority
group in BSF was African-American couples, and the largest minority group in
SHM was Hispanic/Latino couples. (Because of the nature of the studies and the
program sites, there was a relatively small percentage of Hispanic/Latino
couples in BSF and a relatively small percentage of African-American couples in
SHM; hence, the analyses for differential impact focused on the larger groups
within each study.)
For
the earlier assessment points in both BSF and SHM (15 and 12 months,
respectively), there was evidence that the minority couples in the intervention
groups received more benefit than other couples. That is, in BSF, African-American
couples benefitted more than other couples.[ix]
In SHM, Hispanic couples benefitted more
than other couples.[x]
I do not wish to exaggerate these
findings in any way, but the pattern was found in both studies. However, the pattern
did not hold up at the longer-term assessments in either study (36 months for
BSF and 30 months for SHM).
Overall,
these findings suggest that minority couples may have responded relatively more
positively to the programs, on average, than other couples. That some positive
effects fade is not a particularly unusual finding in studies of social
interventions. I believe these findings may suggest an important, positive
response to the interventions but also portray the need for something more and
something continuing. An important question for the field lies in figuring out
what those “somethings” look like in the lives of those interested in, and
responsive to, such efforts.
A Related Finding on
Ethnicity from Another Major Data Set
In
one of our studies, we find an even more striking finding than what was found
in BSF and SHM regarding impacts for ethnic minority couples. This particular
study, funded by NIH,[xi]
would be the single largest randomized trial in the history of the relationship
education field if it were not for BSF and SHM. We have been evaluating the
impact of a version of the intervention we have developed, refined, and tested
over many years, called PREP[xii]
(the Prevention and Relationship Education
Program). Adaptations of PREP were also used in some of the sites in the BSF
and SHM studies.
My
colleagues and I have worked with all branches of the military over the years.
We have worked most closely with military Chaplains, who have a strong
tradition of providing various relationship education services to military
families. In our most recent paper from this project, we present analyses of impacts
at two years following program delivery by U.S. Army chaplains. This paper is
forthcoming in the same journal as Paul Amato’s paper mentioned above.[xiii]
While we had found modest evidence of positive impacts on relationship quality post-intervention,
two years later we found no evidence of sustained impacts on relationship
quality. On the other hand, at the same two-year follow-up, we found that couples
assigned to the intervention were significantly less likely to have divorced than
couples in the control group.[xiv]
This result has some parallels to the BSF results for Oklahoma, with some relationship
quality impacts earlier on and a stability impact later on.[xv]
More
to the purpose here, we found that minority couples received a far larger
divorce reduction impact from the intervention than non-minority couples. Minority
couples in the intervention group were about one-fourth as likely to divorce by the two-year point as minority
couples in the control group. We also found a trend suggesting that couples who
felt the most economic strain had larger divorce reduction impacts, and this economic
strain effect was independent from the minority effect. Such positive impacts
may well fade with longer-term follow-ups (or other positive impacts may emerge),
but the existing findings at two years were striking in the degree to which
minority couples received the greatest benefit in terms of divorce reduction. This,
too, is good news, and it adds to the accumulating evidence that ethnic
minority couples benefit at least as much, and sometimes more, from
relationship education services as do other couples.
Research
in this field marches on. Amidst the ongoing concerns and arguments, I believe there
is some good news to consider as the field continues studying how to foster relationship
stability and quality, both in general and specifically with those individuals
and families who face great disadvantages. I believe it is good news that the
Administration of Children and Families is moving systematically on a program
of research to support increasing effectiveness in family-strengthening efforts.
*
Disclosure: I (along with many colleagues such as Howard
Markman) have played a substantial role in the development of a variety of
relationship education approaches that were used in some of the sites in the
BSF and SHM studies, and that are also used in a number of the projects funded
by the government around the U.S. I receive income from our company called
PREP. Further, I have been a long-time adviser for the efforts in Oklahoma. Of
greater weight for me is the fact that I do believe in trying to build on
promising studies, practices, and program models in the areas I focus on here.
You are entitled, of course, to disregard any of my viewpoints based on these
facts, but I hope those with serious interest would grapple with the ideas and
consider where they may have inherent merit.
[i] Johnson,
M. D. (2012). Healthy Marriage Initiatives: On the need for empiricism in
policy implementation. American Psychologist, 67(4), 296-308. (http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2012-08242-001);
See also: Hawkins, A. J., Stanley, S. M., Cowan, P. A., Fincham, F. D., Beach,
S. R. H., Cowan, C., Rhoades, G. K., Markman, H. J., & Daire, A. P. (2013).
A more optimistic perspective on government-supported marriage and relationship
education programs for lower income couples: Response to Johnson (2012).
American Psychologist, 68(2), 110-111. (http://psycnet.apa.org/journals/amp/68/2/110?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+apa-journals-amp+%28American+Psychologist%29)
[ii] Devaney,
B., & Dion, R. (2010). 15-Month impacts of Oklahoma's Family Expectations
Program. Washington DC: Mathematica Policy Research. (http://www.mathematica-mpr.com/publications/redirect_PubsDB.asp?strSite=PDFs/Family_support/BSF_15month_impacts.pdf)
[iii] This
finding is in the final report for the BSF analyses at 36 months. P. 29 “At the
three-year follow-up, 49 percent of BSF children in Oklahoma had lived with
both parents continuously, compared with 41 percent of children in the control
group (Table A.7b).” An 8% difference over 41% for control group is a 20%
increase. Citation: Wood, R. G., Moore, Q., Clarkwest, A., Killewald, A., &
Monahan, S. (2012). The long-term effects of Building Strong Families: A
relationship skills education program for unmarried parents. Administration for
Children and Families, Office of Planning, Research, and Evaluation. Washington D. C. (http://www.acf.hhs.gov/sites/default/files/opre/bsf_36_mo_impact_report.pdf)
[iv] http://family-studies.org/can-we-strengthen-marriages-results-of-the-supporting-healthy-marriage-evaluation/
[v]
For example: Wood, R. G., Moore, Q., Clarkwest, A., Killewald, A., &
Monahan, S. (2012). The long-term effects of Building Strong Families: A
relationship skills education program for unmarried parents. Administration for
Children and Families, Office of Planning, Research, and Evaluation. Washington D. C. (http://www.acf.hhs.gov/sites/default/files/opre/bsf_36_mo_impact_report.pdf);
Lundquist, E. Hsueh, J., Lowenstein, A. E., Faucetta, K., Gubits, D., Michalopoulos,
C., & Knox, V. (2014). A family-strengthening program for low-income
families: Final impacts from the Supporting Healthy Marriage evaluation. OPRE
Report 2014-09A. Washington, DC: Office of Planning, Research and Evaluation,
Administration for Children and Families, U.S. Department of Health and Human
Services. (http://www.acf.hhs.gov/programs/opre/resource/the-supporting-healthy-marriage-evaluation-a-family-strengthening-program-for-low-income-families-final-impacts-from-the)
[vi] Amato,
Paul R. (2014). Does social and economic disadvantage moderate the effects of
relationship education on couples? An analysis of data from the 15-month
Building Strong Families evaluation. Family Relations, 63, 343-355. doi:
10.1111/fare.12069. (http://onlinelibrary.wiley.com/doi/10.1111/fare.12069/abstract)
[vii] Rauer,
A. J., Adler-Baeder, F., Lucier-Greer, M., Skuban, E., Ketring, S. A., &
Smith, T. (2014). Exploring Processes of Change in Couple Relationship
Education: Predictors of Change in Relationship Quality. Journal of Family
Psychology, 28(1), 65-76. (http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2014-04342-001);
Stanley, S. M., Amato, P. R., Johnson, C. A., & Markman, H. J. (2006).
Premarital education, marital quality, and marital stability: Findings from a
large, random household survey. Journal of Family Psychology, 20(1), 117-126. (http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2006-03561-013)
[viii]
Cowan, P., Cowan, C., Pruett, M., Pruett, K., & Wong, J. (2009). Promoting
fathers' engagement with children: Preventive interventions for low-income
families. Journal of Marriage & Family, 71(3), 663-679. (http://onlinelibrary.wiley.com/doi/10.1111/j.1741-3737.2009.00625.x/abstract)
[ix]
See page xv: Wood, R. G., McConnell, S., Moore, Q., Clarkwest, A., & Hsueh,
J. (2010). The Building Strong Families Project. Strengthening unmarried
parents' relationships: The early impacts of Building Strong Families.
Administration for Children and Families, Office of Planning, Research, and Evaluation.
Washington D. C. (http://www.acf.hhs.gov/programs/opre/strengthen/build_fam/reports/unmarried_parents/15_impact_exec_summ.pdf)
[x]
See page ES-7: Hsueh, J., Alderson, D. P., Lundquist, E., Michalopoulos, C.,
Gubits, D., Fein, D., & Knox, V. (2012). The Supporting Healthy Marriage
Evaluation: Early Impacts on Low-Income Families. Administration for Children and Families,
Office of Planning, Research, and Evaluation.
Washington D. C. (http://www.acf.hhs.gov/sites/default/files/opre/early_impacts_low.pdf)
[xi] This
project is supported by the Eunice Kennedy Shriver National Institute of Child
Health & Human Development of the National Institutes of Health under Award
Number R01HD048780. My comments here are solely my own responsibility and do
not represent any official views of the National Institutes of Health.
[xii]
The actual intervention manuals and materials are not available on the web but
the general principles in PREP are most easily accessible in various books we
have published, e.g.: Markman, H. J., Stanley, S. M., & Blumberg, S. L.
(2010). Fighting for Your Marriage.
San Francisco: Jossey-Bass.
[xiii]
You can write to me to request a copy of the forthcoming paper if you wish. The
citation is: Stanley, S. M., Rhoades, G. K., Loew, B. A., Allen, E. S., Carter,
S., Osborne, L. J., Prentice, D., & Markman, H. J. (in press). A randomized controlled trial of relationship
education in the U.S. Army: 2-year outcomes. Family Relations.
[xiv] The
divorce reduction impact held for the pooled analysis, but it was clearly
driven by the larger of two sites—a site that was comprised of units much more
involved in combat operations and high operational tempo, which also had
younger married couples. The divorce reduction impact was non-existent for a
smaller site where couples were older, more established, and not similarly as
involved in major combat operations. Again, this is consistent with studies in
the field where, when a difference emerges, couples at higher risk tend to get
greater benefits from such services.
[xv]
It is well recognized in this and other fields that one type of positive result
can influence the odds of obtaining a different type of positive result when one result (divorce) causes
people to be missing for analysis of the other outcome (relationship quality). Researchers
at Mathematica (the company that conducted the BSF evaluation) have written a
paper on the depth of the challenges involved in resolving this dilemma in
outcome studies. There is nothing approaching an ideal or perfect solution
because data that are missing for meaningful reasons related to the goals of an
intervention are simply not replaceable. See: McConnell, S., Stuart, E. A.,
& Devaney, B. (2008). The
Truncation-by-Death Problem: What to do in an experimental evaluation when the
outcome is not always defined. Evaluation Review, 37(1), 157-186. (http://erx.sagepub.com/content/32/2/157.full.pdf)
