Some Good News in Who Benefits from Family-Strengthening Programs

Whatever your political views, you likely share in concerns many hold over the difficulties facing socially and economically disadvantaged families in the U.S. But can the government do anything to directly help such families through family-strengthening efforts? Despite all you might have heard to date, there is some good news emerging from recent studies and my goal here is to describe that news.     

The U.S. Administration for Children and Families, specifically the Office of Family Assistance (OFA), is currently invested in three specific components of family strengthening, including efforts to a) improve the quality and stability of the relationships of couples with children, b) increase father involvement for those with fragile relationships with their children, and c) increase the quality of co-parenting between adults who have children in common but who are no longer in an ongoing relationship. My focus here is on research on the first of these types of efforts, as there has been considerable attention to the results of two major federal evaluations of programs aimed to help couples with low incomes and other disadvantages.

The large, multi-site studies were called Building Strong Families (BSF) and Supporting Healthy Marriage (SHM). I believe these studies received so much attention, in part, because they were connected with the somewhat controversial, government-funded efforts over the last twelve years to provide relationship education in support of the goals in the welfare reform law of 1996 to promote two-parent families. These two studies produced disappointing results, which have been lamented among those who support such efforts and trumpeted by those who are critical of such efforts. I think the trumpets have mostly carried the day.

But there is some good news for supporters of relationship education in recent findings, including within those two federal studies. Specifically, it is becoming clear that ethnic minority couples benefit at least as much or more than other couples from such programs. Some have suggested that this would not be the case because such programs were originally designed and tested with mostly middle-class, white couples.[i] Further, there is newly published evidence that, within the BSF study data set, the participants who were the most socially and economically disadvantaged benefitted the most in terms of impacts on relationship quality.

Before describing in more detail what I see as relatively encouraging, I will first describe a bit of background on these important federal studies and their findings to provide context for those who know little about them.

Building Strong Families (BSF) was a study of unmarried couples in the transition to parenthood, whereas Supporting Healthy Marriage (SHM) was a study of married couples. Both projects focused on couples at lower incomes, generally under 200 percent of the poverty line. In both cases, couples in multiple cities were randomly assigned to either receive or not receive a substantial program of relationship education and other couple support services, all designed to strengthen these families with regard to couples’ relationship quality, child outcomes (including father involvement), and stability. While there are some disagreements about the results, it is fair to say that the overall evidence suggested largely non-significant (BSF) or small (SHM) program impacts.

As a way to provide a bit more detail, allow me to give two brief summaries of the findings, as if from the perspectives of people who were either more or less encouraged by the results.  

             Less encouraged. The overall findings for BSF showed no evidence of positive impacts on couples’ relationships and father involvement. There were even some modest negative impacts in some sites. While there was a modest positive impact on child behavior, there were no other overall positive effects on child wellbeing. For SHM, while there was a range of statistically significant positive impacts, the impacts were modest. Further, these particular programs were expensive—much more so than most all historical efforts involving relationship education. In both cases, there was precious little evidence of positive impacts related to child outcomes. There were some very tiny positive impacts for children in the SHM study, but tiny is tiny.

             More encouraged. In the BSF study, one site (Oklahoma) did a particularly impressive job of getting couples in and through services—far outperforming other sites in this respect. The other sites did not, for the most part, get a lot of couples through much of the planned services. Across the whole study, only 55 percent of couples attended any of the relationship education services. Only Oklahoma demonstrated a range of significant and positive impacts on couple relationship outcomes at the 15-month assessment.[ii] While these impacts faded at the 36-month assessment, the children born to couples in the program group were 20% more likely than children born to couples in the control group to have lived continuously with both parents until that 3-year point—also, only in the Oklahoma site.[iii] In SHM, couples showed statistically significant gains at the 12-month assessment and these gains, while small, were largely maintained out to the 30-month assessment. In a field where most policy evaluations of social programs show no significant, lasting impacts, some see this as promising even as the need for improvement is obvious.[iv]

That’s the skinny version of what happened. There are detailed reports and endless commentary on the internet, if you want more information.[v] The dominant story across the media about these studies is that nothing worked. However, there was some good news in how SHM sites learned from experiences in the BSF study, and thus achieved far greater participation and follow-through among couples than BSF sites did. That is, by including strategies to reduce barriers to participation and reinforce attendance, SHM enabled more disadvantaged couples to attend a substantial amount of program services. That is encouraging.

Before moving on, I should mention that there is a whole world of research on relationship education that I am not attempting to cover here, with studies showing generally stronger, positive effects. Further, some experts in that field have been dismayed that so much attention has been focused on BSF and SHM in recent years. I will, however, retain a similarly narrow focus in order to cover findings related to social and economic disadvantage.

The Response of the Most Disadvantaged Couples

To my knowledge, with one exception, the only analyses done to date on the BSF data set have been conducted by the professional evaluation team hired by the Administration for Children and Families to conduct the study. The exception lies in analyses conducted by Paul Amato at Penn State. Amato has approved access to the BSF data set out to the 15-month assessment point, and he has just published a paper with important analyses from that data set.[vi]

Amato sought to assess whether couples at greater disadvantage received more, similar, or less benefits in BSF than other, less disadvantaged couples did. He analyzed outcomes related to both relationship stability (whether couples broke up) and relationship quality.  His method, which was different from any other analysis I have seen to date in this field, was to create a “disadvantage index” based on eleven factors in order to assess whether having a high or low score on this index affected how much couples benefited from the programs. I will quote from Amato’s paper regarding the list of factors going into this index (p. 347):

(a) the mother was less than 20 years old,

(b) the father was less than 20 years old,

(c) the mother did not have a high school degree,

(d) the father did not have a high school degree,

(e) the father was unemployed at baseline,

(f) the father earned less than $10,000 in the last year,

(g) the mother received public assistance in the last year (TANF, food stamps, Medicaid, SCHIP, SSI, SSDI, or WIC),

(h) the mother had one or more children from a previous relationship,

(i) the father had one or more children from a previous relationship,

(j) the mother or father reported no one to care for the baby in an emergency (excluding the partner),

(k) the mother or father reported no one to borrow money from in an emergency (excluding the partner). I [Amato] omitted mothers’ income from the index because the majority of mothers were not in the labor force.

While I need to skip over some technical detail here, I want to note that Amato approached the analyses in a particularly robust way. He tested his findings for what we call sensitivity to various specifications. Essentially, he tried the analyses with various sites left in or out and with various indicators of disadvantage in or out. The findings were robust across such tests.

Before testing for the program impacts, Amato found what one would expect: “The relationship risk variable revealed that higher disadvantage scores were associated with less support and affection, more destructive conflict, less constructive conflict, less trust in partner, more intimate violence, and lower overall relationship quality” (p. 350). Thus, the risk index captures risk as it was designed to do. The important question is, did their level of disadvantage matter for how much benefit couples received in the BSF intervention model? While disadvantage level did not matter for how the program affected couples’ stability (their odds of breaking up), when it came to relationship quality, those with more disadvantage received the most positive impact from the programs.

Quoting from Paul Amato’s paper: 

One of the major criticisms of BSF programs for unmarried couples (and federally funded marriage education programs for low-income couples) is that educational interventions are not effective for disadvantaged populations (Johnson, 2012; Karney & Bradbury, 2005). It is reasonable to imagine that poor couples are so overwhelmed by financial problems and everyday stress that they are unresponsive to relationship education programs and see them as largely irrelevant to their lives. If this were the case, then the most disadvantaged couples – those most at risk of relationship problems – would receive the least benefit from programs like BSF. This study, however, suggests the opposite: Contrary to the notion that disadvantaged couples do not benefit from relationship education, these couples may be the main beneficiaries of these services, provided that they are able to keep their unions intact. (p. 353)

Keep in mind, these results are for the 15 month follow-up. It is possible that if the same analyses are one day repeated for the 36 month follow-up, this same result would not be found. It is not unusual in this field to find impacts in an earlier period that fade by the time a later follow-up is conducted.

In contrast to these encouraging findings from Amato’s paper, results from a meta-analysis working its way toward publication suggest that the very poorest couples receive the least benefit from such programs. (I have the author’s permission to mention what I know about the analyses.)  I believe, however, that the type of analysis in this other study is far less sensitive to addressing the question Amato tested. Nevertheless, the findings from this other study align more closely with the arguments made by the researchers noted in Amato’s quote above, who have suggested that severe economic hardship may interfere with couples’ ability to benefit from such efforts. It is not hard to imagine that chaos and stress would interfere with learning new strategies in one’s relationship. On the other hand, when studies in this field do analyze whether impacts vary based on levels of prior risk, those at greater risk often get the most benefit. There is a lot of complexity here for researchers in the field to sort out.

Amato’s analyses are serious and thoughtful, and he obtained a potentially important finding that is not at all evident from the primary analyses conducted with the BSF data set. That takes nothing away from the main results in BSF (pooled across sites) that are legitimately disappointing for reasons about which serious people may not agree. But Amato’s analyses are encouraging, and perhaps even provocative, for suggesting that such services may actually provide the most benefit, on average, to couples with some of the greatest disadvantages in life. In fact, Amato goes so far as to imply that if the BSF study had recruited substantially more disadvantaged couples, the overall findings across the study would have been positive (p. 353).  

Amato’s findings are not unprecedented. They are the most sophisticated version of a type of finding that has been obtained before, wherein those who are more disadvantaged receive at least as much, and sometimes more, benefit from relationship education services than others.[vii] Amato notes that this is generally the case for various social programs (p. 353). What he found is also consistent with other studies focused on family strengthening that find positive impacts for programs given to highly disadvantaged couples and families. For example, Phil and Carolyn Cowan and their colleagues have demonstrated positive impacts from a program focused on father involvement in a study with low-income families, with a particularly large representation of Mexican American families (67 percent of participants). They found significant, positive impacts on couple relationship quality, father engagement, and children’s problem behaviors.[viii]

The Response of Ethnic Minority Couples

On to other encouraging news I want to share. In both the BSF and SHM studies, the evaluators were able to examine if the minority group with the largest representation got more or less impact than other couples. The largest minority group in BSF was African-American couples, and the largest minority group in SHM was Hispanic/Latino couples. (Because of the nature of the studies and the program sites, there was a relatively small percentage of Hispanic/Latino couples in BSF and a relatively small percentage of African-American couples in SHM; hence, the analyses for differential impact focused on the larger groups within each study.)

For the earlier assessment points in both BSF and SHM (15 and 12 months, respectively), there was evidence that the minority couples in the intervention groups received more benefit than other couples. That is, in BSF, African-American couples benefitted more than other couples.[ix]  In SHM, Hispanic couples benefitted more than other couples.[x]  I do not wish to exaggerate these findings in any way, but the pattern was found in both studies. However, the pattern did not hold up at the longer-term assessments in either study (36 months for BSF and 30 months for SHM).

Overall, these findings suggest that minority couples may have responded relatively more positively to the programs, on average, than other couples. That some positive effects fade is not a particularly unusual finding in studies of social interventions. I believe these findings may suggest an important, positive response to the interventions but also portray the need for something more and something continuing. An important question for the field lies in figuring out what those “somethings” look like in the lives of those interested in, and responsive to, such efforts.

A Related Finding on Ethnicity from Another Major Data Set

In one of our studies, we find an even more striking finding than what was found in BSF and SHM regarding impacts for ethnic minority couples. This particular study, funded by NIH,[xi] would be the single largest randomized trial in the history of the relationship education field if it were not for BSF and SHM. We have been evaluating the impact of a version of the intervention we have developed, refined, and tested over many years, called PREP[xii]  (the Prevention and Relationship Education Program). Adaptations of PREP were also used in some of the sites in the BSF and SHM studies.

My colleagues and I have worked with all branches of the military over the years. We have worked most closely with military Chaplains, who have a strong tradition of providing various relationship education services to military families. In our most recent paper from this project, we present analyses of impacts at two years following program delivery by U.S. Army chaplains. This paper is forthcoming in the same journal as Paul Amato’s paper mentioned above.[xiii] While we had found modest evidence of positive impacts on relationship quality post-intervention, two years later we found no evidence of sustained impacts on relationship quality. On the other hand, at the same two-year follow-up, we found that couples assigned to the intervention were significantly less likely to have divorced than couples in the control group.[xiv] This result has some parallels to the BSF results for Oklahoma, with some relationship quality impacts earlier on and a stability impact later on.[xv]

More to the purpose here, we found that minority couples received a far larger divorce reduction impact from the intervention than non-minority couples. Minority couples in the intervention group were about one-fourth as likely to divorce by the two-year point as minority couples in the control group. We also found a trend suggesting that couples who felt the most economic strain had larger divorce reduction impacts, and this economic strain effect was independent from the minority effect. Such positive impacts may well fade with longer-term follow-ups (or other positive impacts may emerge), but the existing findings at two years were striking in the degree to which minority couples received the greatest benefit in terms of divorce reduction. This, too, is good news, and it adds to the accumulating evidence that ethnic minority couples benefit at least as much, and sometimes more, from relationship education services as do other couples.

Research in this field marches on. Amidst the ongoing concerns and arguments, I believe there is some good news to consider as the field continues studying how to foster relationship stability and quality, both in general and specifically with those individuals and families who face great disadvantages. I believe it is good news that the Administration of Children and Families is moving systematically on a program of research to support increasing effectiveness in family-strengthening efforts.   

*

Disclosure:  I (along with many colleagues such as Howard Markman) have played a substantial role in the development of a variety of relationship education approaches that were used in some of the sites in the BSF and SHM studies, and that are also used in a number of the projects funded by the government around the U.S. I receive income from our company called PREP. Further, I have been a long-time adviser for the efforts in Oklahoma. Of greater weight for me is the fact that I do believe in trying to build on promising studies, practices, and program models in the areas I focus on here. You are entitled, of course, to disregard any of my viewpoints based on these facts, but I hope those with serious interest would grapple with the ideas and consider where they may have inherent merit.

---

[i] Johnson, M. D. (2012). Healthy Marriage Initiatives: On the need for empiricism in policy implementation. American Psychologist, 67(4), 296-308. (http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2012-08242-001); See also: Hawkins, A. J., Stanley, S. M., Cowan, P. A., Fincham, F. D., Beach, S. R. H., Cowan, C., Rhoades, G. K., Markman, H. J., & Daire, A. P. (2013). A more optimistic perspective on government-supported marriage and relationship education programs for lower income couples: Response to Johnson (2012). American Psychologist, 68(2), 110-111. (http://psycnet.apa.org/journals/amp/68/2/110?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+apa-journals-amp+%28American+Psychologist%29)

[ii] Devaney, B., & Dion, R. (2010). 15-Month impacts of Oklahoma's Family Expectations Program. Washington DC: Mathematica Policy Research. (http://www.mathematica-mpr.com/publications/redirect_PubsDB.asp?strSite=PDFs/Family_support/BSF_15month_impacts.pdf)

[iii] This finding is in the final report for the BSF analyses at 36 months. P. 29 “At the three-year follow-up, 49 percent of BSF children in Oklahoma had lived with both parents continuously, compared with 41 percent of children in the control group (Table A.7b).” An 8% difference over 41% for control group is a 20% increase. Citation: Wood, R. G., Moore, Q., Clarkwest, A., Killewald, A., & Monahan, S. (2012). The long-term effects of Building Strong Families: A relationship skills education program for unmarried parents. Administration for Children and Families, Office of Planning, Research, and Evaluation.  Washington D. C. (http://www.acf.hhs.gov/sites/default/files/opre/bsf_36_mo_impact_report.pdf)

[iv] http://family-studies.org/can-we-strengthen-marriages-results-of-the-supporting-healthy-marriage-evaluation/

[v] For example: Wood, R. G., Moore, Q., Clarkwest, A., Killewald, A., & Monahan, S. (2012). The long-term effects of Building Strong Families: A relationship skills education program for unmarried parents. Administration for Children and Families, Office of Planning, Research, and Evaluation.  Washington D. C. (http://www.acf.hhs.gov/sites/default/files/opre/bsf_36_mo_impact_report.pdf); Lundquist, E. Hsueh, J., Lowenstein, A. E., Faucetta, K., Gubits, D., Michalopoulos, C., & Knox, V. (2014). A family-strengthening program for low-income families: Final impacts from the Supporting Healthy Marriage evaluation. OPRE Report 2014-09A. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. (http://www.acf.hhs.gov/programs/opre/resource/the-supporting-healthy-marriage-evaluation-a-family-strengthening-program-for-low-income-families-final-impacts-from-the)

[vi] Amato, Paul R. (2014). Does social and economic disadvantage moderate the effects of relationship education on couples? An analysis of data from the 15-month Building Strong Families evaluation. Family Relations, 63, 343-355. doi: 10.1111/fare.12069. (http://onlinelibrary.wiley.com/doi/10.1111/fare.12069/abstract)

[vii] Rauer, A. J., Adler-Baeder, F., Lucier-Greer, M., Skuban, E., Ketring, S. A., & Smith, T. (2014). Exploring Processes of Change in Couple Relationship Education: Predictors of Change in Relationship Quality. Journal of Family Psychology, 28(1), 65-76. (http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2014-04342-001); Stanley, S. M., Amato, P. R., Johnson, C. A., & Markman, H. J. (2006). Premarital education, marital quality, and marital stability: Findings from a large, random household survey. Journal of Family Psychology, 20(1), 117-126. (http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2006-03561-013)

[viii] Cowan, P., Cowan, C., Pruett, M., Pruett, K., & Wong, J. (2009). Promoting fathers' engagement with children: Preventive interventions for low-income families. Journal of Marriage & Family, 71(3), 663-679. (http://onlinelibrary.wiley.com/doi/10.1111/j.1741-3737.2009.00625.x/abstract)

[ix] See page xv: Wood, R. G., McConnell, S., Moore, Q., Clarkwest, A., & Hsueh, J. (2010). The Building Strong Families Project. Strengthening unmarried parents' relationships: The early impacts of Building Strong Families. Administration for Children and Families, Office of Planning, Research, and Evaluation. Washington D. C. (http://www.acf.hhs.gov/programs/opre/strengthen/build_fam/reports/unmarried_parents/15_impact_exec_summ.pdf)

[x] See page ES-7: Hsueh, J., Alderson, D. P., Lundquist, E., Michalopoulos, C., Gubits, D., Fein, D., & Knox, V. (2012). The Supporting Healthy Marriage Evaluation: Early Impacts on Low-Income Families.  Administration for Children and Families, Office of Planning, Research, and Evaluation.  Washington D. C. (http://www.acf.hhs.gov/sites/default/files/opre/early_impacts_low.pdf)

[xi] This project is supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health under Award Number R01HD048780. My comments here are solely my own responsibility and do not represent any official views of the National Institutes of Health.

[xii] The actual intervention manuals and materials are not available on the web but the general principles in PREP are most easily accessible in various books we have published, e.g.: Markman, H. J., Stanley, S. M., & Blumberg, S. L. (2010). Fighting for Your Marriage. San Francisco: Jossey-Bass.

[xiii] You can write to me to request a copy of the forthcoming paper if you wish. The citation is: Stanley, S. M., Rhoades, G. K., Loew, B. A., Allen, E. S., Carter, S., Osborne, L. J., Prentice, D., & Markman, H. J. (in press).  A randomized controlled trial of relationship education in the U.S. Army: 2-year outcomes. Family Relations.

[xiv] The divorce reduction impact held for the pooled analysis, but it was clearly driven by the larger of two sites—a site that was comprised of units much more involved in combat operations and high operational tempo, which also had younger married couples. The divorce reduction impact was non-existent for a smaller site where couples were older, more established, and not similarly as involved in major combat operations. Again, this is consistent with studies in the field where, when a difference emerges, couples at higher risk tend to get greater benefits from such services.

[xv] It is well recognized in this and other fields that one type of positive result can influence the odds of obtaining a different type of positive  result when one result (divorce) causes people to be missing for analysis of the other outcome (relationship quality). Researchers at Mathematica (the company that conducted the BSF evaluation) have written a paper on the depth of the challenges involved in resolving this dilemma in outcome studies. There is nothing approaching an ideal or perfect solution because data that are missing for meaningful reasons related to the goals of an intervention are simply not replaceable. See: McConnell, S., Stuart, E. A., & Devaney, B. (2008).  The Truncation-by-Death Problem: What to do in an experimental evaluation when the outcome is not always defined. Evaluation Review, 37(1), 157-186. (http://erx.sagepub.com/content/32/2/157.full.pdf) 

About Those Controversial “Marriage” Programs Funded by the Government

The fur is flying once again around questions of the wisdom and effectiveness of the government funded efforts to increase and strengthen two parent families through the use of relationship education strategies. I believe that things have heated up, in part, because this issue about relationship education has now intersected with this year’s political debate about the nature of poverty and income inequality. That is too bad because politically-infused rhetoric is not conducive to the type of smart policy discussions that need to be sustained to actually make a difference in the lives of real people.

In this post, I am going to link to four articles that exemplify the issues being debated right now. My goal here is to focus in on a few specific points raised in the four articles, particularly related to the subject of government funded initiatives providing relationship education. Links are provided if you want to find and read the pieces I reference.

First up, Kay Hymowitz wrote a broad, thoughtful piece entitled How Single Motherhood Hurts Kids (New York Times on February 8^th, 2014). She covers a lot of important ground. I like how she challenges some of the deeply held viewpoints of liberals and conservatives. I also like how she embraces the bi-directionality of how poverty impacts families and how romantic and sexual relationships impact poverty. I think, for some people, the latter point may get uncomfortably close to how marriage might play some causal role in poverty. For example, romantic and sexual relationships impact childbearing and also the odds of succeeding in marriage, which, in turn, all influence poverty. I believe that discussions about poverty and family policy/programs that do not embrace the obvious and complex ways that causation flows both ways are unproductive; it is better to embrace than ignore complexity when it is relevant.

Next up is a piece by Annie Lowrey, in the New York Times Magazine: Can Marriage Cure Poverty? (February 4^th, 2014). Lowrey’s piece is excellent though I think it is slightly marred by the way she caricatures conservatives. But she gets some crucial points right. First, non-marriage is linked with poverty in some profound ways, and the only real debate on that subject is not if but how. Second, she acknowledges that there is some positive news in the recent, large-scale government funded evaluations testing relationship education strategies with unmarried couples in poverty. Third, she directly addresses the topic of the unbelievably dumb disincentives within government policies that punish disadvantaged couples as they try to get ahead. That’s a topic for another day, but the disincentives are probably growing and that makes me fear for our future.

On the other hand, Lowrey tilts pretty far to the unidirectional viewpoint that poverty drives problems in family structure and marriage without addressing the ways romantic partnering and sexual relationships contribute to disadvantage and risk. Hymowitz, as I noted above, digs in on this important issue while also addressing aspects of the complexity of marriage for some among the disadvantaged.

I raise these particular issues here because I want draw attention to one type of relationship education that has been utilized in some of the government supported efforts that gets little or no attention in the typical discussions. There have been a lot of services focused on helping individuals, not couples, make relationship choices that are designed to foster their odds of achieving their own life goals (including, eventually, marriage if desired) while also improving the lives of their children. Such individually-oriented relationship education is founded on the awareness that the economic, mental, and physical wellbeing of adults is deeply impacted by what happens in their love lives. Anyone close to the challenges confronting those with immense disadvantages recognizes that landing with a wrong, poorly matched, or dangerous partner can completely upend any other progress in life. Does any of this type of work get acknowledged in public discourse about government funded relationship education? Rarely. On the ground, I see liberals and conservatives united around the obvious rationality of such strategies. We do need robust evaluations of such services (they are pretty new and innovative), but the goals are important and some of the work is immensely thoughtful. 

Instead of showing any awareness of some of the innovative work being attempted, what is often suggested or implied in media stories is that government funded relationship education efforts are focused primarily on getting people in poverty to get partnered-up in marriage because that will magically cure all the ills of poverty, family functioning, and individual wellbeing. Right. Of course it will. And, of course, that’s how most of the people working on the ground in such efforts think. Magic beans, Jack.

If your sarcasm meter is not functioning, let me clarify: I’ve been around this work for a long time. I do not know anyone deeply involved who thinks so simplistically about this work. In fact, on the broader subject of thoughtfulness, I have been fortunate to be in scores of meetings with diverse scholars and experts grappling with complex issues about helping people—meetings where what happens is a lot more substantive than implied, for example, by a critic in one of the upcoming pieces I link to below who refers to “mucking around in people's lives.”

Related to complexity and thoughtfulness, I also want to suggest that anyone who wants to be taken seriously in discussing relationship education and government policies needs to think carefully about the types of outcomes that would be positive in different situations. For example, there are many cases wherein a good outcome of relationship education amounts to a damaging or destructive relationship coming to an end—not ending in marriage. Marriage is a public and private good, desired by many, but that does not mean any marriage to anybody. Strategists and workers on the ground level seem to have wholly accepted the goal of fostering healthy relationships and marriage; this fact seems to elude most critics of the efforts. This point also highlights just one of the reasons why it may be quite challenging to interpret simple statistics about marriage and divorce rates at the macro level as part of these discussions. 

Moving on to the other two articles. Both deal evidence of effectiveness of services provided to couples in some of the most intensive type of government efforts, with a particular focus on two large, federally funded, multi-site studies of program impacts for comprehensive services provided to disadvantaged couples who are either unmarried and having a child (The Building Strong Families Study: BSF) or married with children (The Supporting Healthy Marriage Study: SHM).

One lengthy piece was written by Thomas Bartlett of The Chronicle of Higher Education: The Great Mom & Dad Experiment (January 20^th, 2014). There are some elements of Bartlett’s piece that I take issue with, but, for this type of policy discussion, the piece seems to me to be relatively fair. He does a good job describing a range of complex issue and areas of serious disagreement among some of those who have thought about these issues. Another piece also covers a good deal of ground. It’s by Emily Alpert Reyes of McClatchy News: Federal Programs to Improve Marriage Don't Work (February 10, 2014). As with others here, there are complex issues raised in this piece that I may come back to in a later piece, but I mention both of these pieces here because I want to wrap up this post by making three points related to those two large government studies noted above. Many commentators of these two studies discuss the results as if there were no impacts at all. Sometimes, modest impacts are acknowledged but with an assumption that, surely, the government would not and should not continue to fund programs in any area where there were similarly modest findings in large, randomized trials.

In making these points, I take nothing from the fact that the actual findings and implications are complex.

1.  The one site among eight in the BSF study (Family Expectations in Oklahoma City) that successfully delivered a serious “dose” of the services to the participating couples demonstrated a range of statistically significant, positive impacts at the 15 month follow-up. Further, there was a statistically significant impact on family stability (a 20% increase) at the 3 year point follow-up. It is true that there was an absence of evidence for other types of significant impacts. However, obtaining a relatively large impact on a core, long-term outcome of policy significance (family stability) seems pretty important. Such an outcome is rare in this type of study.

2. Those conducting the large, SHM study, learned a great deal from the BSF study. As a result, all eight sites in that study delivered a substantial dose of the services to most couples in the program groups. What did they find? The results from the 12 month follow-up showed statistically significant, though small, impacts on a number of variables related to marital quality. (There are findings coming out soon on the 3 year follow-up.)   

3. Most large-scale studies of most government programs find no impacts. In 1987, a sociologist deeply acquainted with government evaluations, Peter Rossi, came up with The Iron Law of Evaluation, which he stated thusly: “The expected value of any net impact assessment of any large scale social program is zero.” Translation? In most such studies, one can expect no significant impacts; yet there were interpretable, significant impacts in the two studies noted above of these early efforts.  

I could list a wide variety of government programs and services that have poor evidence of effectiveness but that, nevertheless, receive vastly greater funding than the nascent attempts to help individuals and couples in their relationships. There are also many smaller studies, including some with very strong methodology, that show consistent evidence of positive impacts from relationship education. (Not my topic today.)

Am I pleased with modest impacts of those two large studies? No. As someone who works in this field, I see the current studies as promising, but like many others, I want to see us learn from current efforts and strengthen impacts.

In all of this, I have been puzzled by calls to defund what seem to me to be, by historical standards, promising findings coming early in a relatively new arena for government involvement. Why don’t the same critics explicitly call for defunding services and programs that receive hundreds of times the funding, services that happen to have long trails of meager or non-significant findings? I cannot read the minds of others, but I think some of the answer comes back to where I started. I think some of the energy currently rushing into this space, but certainly not all, is really about other issues.   

* *

Disclosure:  I think that just about everyone (but not everyone) writing and commenting on these matters has some type of bias or values animating their arguments, and it is worth considering what those may be when thinking through the points being bandied about. I try to be very above board, so here is my disclosure. You should know that Howard Markman and I (and a host of colleagues) have developed a variety of relationship education approaches that were used in some of the sites in the BSF and SHM studies and that are also used in a number of the projects funded by the government around the U.S. I receive income from our company called PREP. Further, I have been a long-time adviser for the efforts in Oklahoma. Of greater weight within me is the fact that I do believe in trying to build on promising studies, practices, and program models in these areas I focus on here. You are entitled, of course, to disregard any of my viewpoints based on these facts, but I hope those with serious interest would grapple with the ideas and consider where there may be inherent merit.

Reference for Rossi:  Rossi, P. H. (1987). The Iron Law of Evaluation and other metallic rules. Research in Social Problems and Public Policy, 4, 3-20.  (for an interesting piece on this, click here) 

Does Relationship/Marriage Education Work with Participants who have Low Incomes and/or with Minority Participants?

Note to usual readers of my blog. On 3-11-13, I posted an unusually long blog entry regarding an ongoing discussion among researchers and policy experts about the usefulness and impacts of relationship education services provided to couples who are economically disadvantaged or to couples of color. A colleague who is also a couple/marital researcher, Matthew Johnson, wrote a piece published last year (2012) in the American Psychologist wherein he suggested that the results of two large federal studies demonstrated that relationship education efforts were ineffective. He suggested that this may have been due, in part, to many of the couples in the studies having characteristics different from couples who have been studied more heavily in this field. Johnson made a reasonable call for more research to inform the field in working with couples who are disadvantaged. However, a number of researchers (including me) took issue with other suggestions and points made by Johnson. While we agreed with the wisdom in the call for more research, we took issue with aspects of Johnson's arguments. For example, we suggested that there was growing evidence that such relationship education interventions may be as--or even more--effective with the very couples Johnson suggested were not benefiting because of inadequate research. 

The long post that I wrote on all of this is now included in a document that can be downloaded. The document includes the links to Johnson's original paper, the comment by a number of other researchers (including me) on his paper, and a reply by Johnson to our comment. The document also includes a link to a very helpful chapter on the subject by Alan Hawkins. My post on all of this is long and technical, and I have now moved it from the main stream of blog entries here to the document that you can download here.

*

Family Stability and Relationship Education Services: A Comment on Cherlin’s Critique

Andrew Cherlin, a sociologist studying families, published an opinion in Bloomberg on 12-25-12, entitled Do Unmarried Poor Have Bad Values or Bad Jobs?  In it, he writes several things I will comment on here.  In the piece, he says: 

“A large U.S. government-funded experiment to encourage low-income parents to marry, a legacy of the George W. Bush administration’s Healthy Marriage Initiative, has just fallen flat. Even if you were a skeptic all along of the wisdom of the government promoting marriage, as I was, this isn’t good news. For the children of these unmarried couples, it is bad news: It portends years of unstable, complicated home lives. The apparent failure of marriage promotion makes the task of finding other ways to help them even more urgent.”

To his point about family instability, I can only say, “Too right.” (Not as in too right or too left but as in very correct.) He covers various points about the costs of family instability to society in his piece, and he notes that children in the U.S. whose parents are married have far more likelihood of family stability than parents of children who cohabit. I have increasingly emphasized family instability because it is a very big deal (see my piece on The Perfect Storm).  As Cherlin writes in his piece, “I also am convinced that children do best in stable family environments and that repeated parental breakups and ‘repartnering’ can be harmful to them.” He is so “onboard” with understanding this societal issue that, before I take issue with his piece, I will first recommend his book, The Marriage-Go-Round (I will even link that to Amazon. Wish my critics would give links to books I’ve written or co-authored!). 

I am bothered by two points in Cherlin’s piece, and I want to give you the other side. 

First, Cherlin contextualizes the type of service being tested as “government promoting marriage.”  He gives a brief history of these efforts that were, over the past 10 years, particularly encouraged by the Bush administration. All such programs stemmed from the welfare reform law of 1996 in which President Clinton and the Republican congress followed through on Clinton’s campaign vow to end welfare as we knew it. Under the foundational principles of the welfare reform act of 1996, states were encouraged to try to increase the formation and stability of two parent families. Few states attempted anything of the like, and the Bush administration subsequently provided funding and grants to increase these efforts.  Some of these types of efforts have been continued under the Obama administration as well. And whatever else you may think of such efforts, I can tell you that a type of service that is historically simply not available to low income families became possible under such initiatives.  

I do not like seeing such efforts described as “promoting marriage.” Cherlin’s piece implies that many of the government supported efforts in this area, of the last decade, focused on the values of low income individuals about marriage. That’s not only not true about programs, it’s not true about the values people already hold. We’ve known well, and for well over a decade, that the very poor are less likely to marry for reasons other than not valuing marriage. In fact, they tend to value it more than others (for example, see Kathy Edin and Maria Kefalas’ book, Promises I Can Keep). 

Here’s my beef, and it’s a large burger with fries and a Coke.  I see this term “marriage promotion” fairly often in journal articles and policy discussions and I think it mischaracterizes what most of these efforts have been about. You would be hard pressed to find any substantial push on those receiving such services to go and get married, as if doing so would magically solve their problems in life. Cherlin rightly describes some of the services (though a bit narrowly I think) as ‘“relationship skills” programs to improve communication, avoid conflict and build trust.’ Such services are not generally new—though there is much new in the services devised to meet the needs of the economically disadvantaged. 

Cherlin goes on to emphasize the interplay between the economy, structural changes in the job market, and the current level of family instability and non-marriage among the economically disadvantaged. These are important, valid points. For an excellent piece on this theme from earlier this year, see the article in the New York Times by Jason DeParle, entitled Two Classes, Divided by ‘I Do’ (also, see earlier blog entry I wrote on this point.)

Economics, relationship quality, and relationship commitment all play dramatic roles in what is unfolding in understanding family instability. And do not forget what I keep coming back to. One of the main reasons why children of unmarried couples, especially those at lower incomes, are much more likely to experience family instability is not that their biological parents are worse parents than marrieds—unmarried couples are simply far less likely to be having a child in the context of an already settled, mutual commitment to the future. The important policy matter is that these couples are much more vulnerable, on average, than other couples.

My second point has to do with the findings of the large, federally funded study that Cherlin discusses. See his piece for more details. The study is called The Building Strong Families Study (BSF).  What Cherlin describes is quite accurate; what he leaves out is too important to have been left out. 

First off, I should point out that this large study (BSF) tested one of the most expensive models of the recent government efforts, and with a particularly challenging population: couples having a child who are economically disadvantaged and who are not married. The rate of family instability for these groups is very high. Here is what is not mentioned in Cherlin’s piece that I think is worth you knowing about.  

This large study included 8 sites, nationally. In only one of these 8 sites did most couple receive any substantial amount of the planned services. In Oklahoma’s Family Expectations Program, a combination of cutting-edge efforts to reduce barriers to participation, reinforcements for attending, and quality of the services led to exceptional attendance. Attendance was dismal across the rest of the sites. And, if you follow any media and opinion pieces on this study (and related issues), you are almost always only being told about the findings averaged across the sites. Only the Oklahoma site had solid results on many dimensions of relationship quality at the 15 months assessment. 

As for the past and recent (3 years out) outcomes, As Cherlin notes in his piece, “Only the Oklahoma site showed some positive effects.” But what is not mentioned is that the Oklahoma site was the only one where there was a statistically significant impact on an important outcome at the 3 year point: in the program group, 49% of the families had lived together continuously since the birth of the child whereas, for the control group, 41% of the families had remained together in this way.

What were we talking about here? Oh yes, family instability. 

I need to explain this, as you might think that 49% compared to 41% does not sound like much of a big deal. That difference in 8 percentage points at three years out amounts to a 20% increase in the likelihood of these families continuously living together if they were in the program group.  And this is a finding on a core measure of family wellbeing (stability). It also reflects something that is rare in large, rigorous studies of government programs. What’s rare? The fact that this impact is statistically significant, it is large, and it is relevant.  You might think this happens all the time in studies of government programs, but I am using the word “rare” deliberately, here. There is an almost unbelievably weak record of lasting impacts in studies of most government supported programs to achieve specific effects. Take jobs training as just one example. The record of effectiveness is quite poor; and I do not mean, by this, to argue against such services.  The dismal evidence means that such programs need to be made more effective than they now are—if possible. [Want to check me out on this? Start here, in this fairly recent GAO report. Try searching for the paragraph that begins with “Little is known” and read that and go from there in your studying of evidence in government programs, if you like. There is no shortage of information on the internet.]

Now, back to this 8% difference in family stability. This finding suggests that if you put 1000 low income, unmarried couples through this program, 80 families (80 two parent families) will be together three years later that otherwise would not have been. As Cherlin notes, this particular type of program is expensive, averaging $11,000 per couple. Much of the services initiated in the past 10 years or as part of the federal initiatives cost far less. Further, these more expensive programs could very likely come down in cost through further testing and refinement. Even if that cost remained that high, consider this math that assumes no improvements in cost efficiencies: 1000 couples treated times $ 11,000 equals total program cost. Divide that product by the 80 families still together who otherwise would not be. That means it cost $ 137,500.00 per family for that stability (and this assumes no other benefits to anyone). That sounds like an awful lot of money, but you don’t have to have be around government family policy discussions very long to suspect that the impacts of early family instability for children have lifetime costs to society well exceeding that amount. And that can be considered a starting point to work from, not the end of that adventure in government policy and programs.  

So, what’s my point? It’s twofold.  Many of the efforts of the last 10 years to increase family stability (and, yes, in marriage where possible) have been routinely mischaracterized. Whatever else is true or not, one of the things that happened these past 10 years is that a type of service usually not remotely available to low income couples became more widely available to them. And there is body of evidence that such strategies are generally effective (click here for more information). In this particular, large government study just reaching the 3 year point (BSF), there was an unusual, statistically significant, long-term result on one of the most important outcomes in this field—and within the only site that got a substantial number of couples into the intervention. I am far from Pollyannaish about all this, but this is worth thinking more about. You are not going to see this point made in most reports on the study because, like as Andrew Cherlin admirably confessed about his own views in his piece, most family policy experts have been skeptics of such efforts. There is something to build on in these findings. 




*
Disclaimer:  I try to be very above board, so here is my disclaimer. You should know that my colleague Howard Markman and I developed a program for couples called PREP which was adapted by Pam Jordan (Becoming Parents), and that was a core part of the total program delivered in Oklahoma. I receive income from PREP. Further, I have been a long-time adviser for the efforts in Oklahoma. Like Andrew Cherlin, I am pretty skeptical of many things in my field. But I clearly see this area of work  as the “glass half full” and not the “glass all empty.” But do not take the word of either of us.  You can check any of these assertions for yourself in numerous reports and pieces on the web. But please set your expectations to start with by looking for compelling evidence of lasting impact for most government programs. Otherwise, you can too easily have an unfair standard of comparison for programs such as the one this piece focuses on.

Who Benefits from Relationship Education? Notes from my Plenary Address at NARME

This is one of those entries that will be of more interest to a specific subgroup of those who follow this blog.  It’s about findings and some controversies in the field of couple relationship education (CRE; also often called marriage and relationship education).  For the rest of you, I promise something fun later next week.

In July, I gave a plenary address at the National Association of Relationship and Marriage Education annual conference in Baltimore.  I had two goals in this talk. 

Goal #1:  My first goal was to present an update of findings from our study of CRE delivered to US Army couples by chaplains.  Some of those findings have already been published but I also presented the findings from the most recent analyses that will go into journal reports we are writing at this time.  My co-investigators in this work are Elizabeth Allen, Howard Markman, & Galena Rhoades, and the study is funded by The Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD). 

Goal #2: My second goal was to address some issues raised in recent discussions about the impact of CRE. I do this by covering some specific findings from three large samples (including the Army study) as well as meta-analyses of an array of studies.  If you are interested in the debates about the government efforts in the past decade that funded some community efforts that used CRE, I cover some of the important issues in my talk.  

You can download a pdf of the slides from my talk at NARME,here. 

You can review my post from last year, summarizing general evidence related to CRE, here.

I’ll try for something edgy on dating and mating in a week and a half or so!

*