- Study protocol
- Open Access
Methods and protocol of a mixed method quasi-experiment to evaluate the effects of a structural economic and food security intervention on HIV vulnerability in rural Malawi: The SAGE4Health Study
SpringerPlus volume 3, Article number: 296 (2014)
Poverty and lack of a predictable, stable source of food are two fundamental determinants of ill health, including HIV/AIDS. Conversely, episodes of poor health and death from HIV can disrupt the ability to maintain economic stability in affected households, especially those that rely on subsistence farming. However, little empirical research has examined if, and how, improvements in people’s economic status and food security translate into changes in HIV vulnerability.
In this paper, we describe in detail the methods and protocol of an academic-NGO collaboration on a quasi-experimental, longitudinal study of the mechanisms and magnitude of the impact of a multilevel economic and food security program (Support to Able-Bodied Vulnerable Groups to Achieve Food Security; SAFE), as implemented by CARE. Primary outcomes include HIV vulnerability (i.e., HIV risk behaviors, HIV infection), economic status (i.e., income, household assets) and food security (including anthropometric measures). We recruited participants from two types of areas of rural central Malawi: traditional authorities (TA) selected by CARE to receive the SAFE program (intervention group) and TAs receiving other unrelated CARE programming (controls). In the intervention TAs, we recruited 598 program participants (398 women, 200 men) and interviewed them at baseline and 18- and 36-month follow-ups; we interviewed 301 control households. In addition, we conducted random surveys (n = 1002) in the intervention and control areas with a 36-month assessment interval, prior to and after implementation of SAFE. Thus, we are examining intervention outcomes both in direct SAFE program participants and their larger communities. We are using multilevel modeling to examine mediators and moderators of the effects of SAFE on HIV outcomes at the individual and community levels and determine the ways in which changes in HIV outcomes feed back into economic outcomes and food security at later interviews. Finally, we are conducting a qualitative end-of-program evaluation consisting of in-depth interviews with 90 SAFE participants.
In addition to examining pathways linking structural factors to HIV vulnerability, this research will yield important information for understanding the impact of a multilevel environmental/structural intervention on HIV, with the potential for other sustainable long-term public health benefits.
HIV/AIDS, poverty, and food insecurity contribute substantially to morbidity and mortality in sub-Saharan Africa (Masanjala 2007; Tsai et al. 2011). The Republic of Malawi, in southeastern Africa, bears one of the heaviest HIV disease burdens globally, ranking ninth worldwide for prevalence of HIV/AIDS among adults (ages 15–49) (Central Intelligence Agency CIA 2010). The 2010 Malawi Demographic and Health Survey (MDHS) (National Statistical Office NSO and ICF Macro 2011) reported an adult HIV prevalence rate of 11%, with a disproportionately higher rate for women than for men (13% versus 8%; (National Statistical Office NSO and ICF Macro 2011). Poverty is endemic in Malawi; more than half of its estimated 15 million people live on less than a dollar a day (Masanjala 2007; Integrated Household Survey IHS3 2010-2011). Food insecurity, defined as having uncertain or limited access to nutritionally adequate food, or being unable to procure food in socially acceptable ways (Anderson 1990; Olson 1999), is an aggravated problem in Malawi. The latest Integrated Household Survey (IHS3, 2010–2011) showed that half of Malawian households (49%) faced food shortages in the year preceding the survey. Taken together, Malawi seems to be caught in a vicious cycle wherein poverty and food insecurity increase people’s vulnerability to HIV infection, while HIV infection reduces peoples’ ability to earn money and produce food (Aarehag et al. 2006; Bryceson et al. 2004; de Waal and Whiteside 2003).
To better understand the context of HIV in Malawi, and to determine potential responses, it is important to consider HIV within an ecosocial framework (Krieger 1994; Loevinsohn and Gillespie 2003). Moving beyond the conventional focus on proximal factors contributing to HIV vulnerability, like individual risk behaviors, it is essential that interventions address poverty and food insecurity (Anema et al. 2009; Miller et al. 2011) as interrelated distal factors in the HIV pandemic, especially in countries like Malawi. Poverty has been consistently recognized as a risk factor for food insecurity (Maes et al. 2010; Nelson et al. 1998; Normén et al. 2005). Focusing on poverty, the Intervention with Microfinance for AIDS and Gender Equity (IMAGE) study, which combined a microfinance program with a gender and HIV training curriculum, was associated with decreased levels of self-reported intimate partner violence and unprotected sex (Kim et al. 2007; Pronyk et al. 2006).
Food insecurity, in turn, is associated with decreased adherence to HIV antiretroviral therapy (Weiser et al. 2009a, b), incomplete virologic suppression (Wang et al. 2011; Weiser et al. 2009a), declines in physical health status, (Weiser et al. 2009a, 2010), worse immunologic status (Kalichman et al. 2010; Weiser et al. 2009b), increased incidence of serious illness (Weiser et al. 2010) increased mortality (Anema et al. 2013) and decreased survival (Anema et al. 2013). In particular, a cross-sectional study in Brazil found that severe food insecurity with hunger is associated with reduced condom use and with increased occurrence of symptoms that may indicate sexually transmitted disease among sexually active women (Tsai et al. 2012). These findings indicate that HIV prevention strategies in food insecure areas should include interventions that target food insecurity.
Increasing critique has targeted the limitations of proximally focused HIV prevention interventions (Weinhardt et al. 1999; Johnson et al. 2003) and pointed out the need for development and assessment of complex, multilayered structural interventions (Gupta et al. 2008) that address root causes and causal pathways linking social, economic, political, and environmental factors to HIV vulnerability in specific contexts (Gupta et al. 2008; Hargreaves 2013). There are significant gaps in knowledge, however, about the development, implementation, and evaluation of structural interventions. First, while integration of food security interventions into HIV/AIDS prevention programs may be essential to curtail the HIV/AIDS pandemic and improve health and quality of life among those infected in resource-poor settings, (Weiser et al. 2011) the literature has offered little guidance to international policy makers, such as the World Food Programme (World Food Programme 2003). To our knowledge, there have been no published intervention studies examining the impact of economic status and food security on HIV outcomes in Malawi. Second, complex multilevel structural interventions are expensive. Typically, non-governmental organizations (NGOs) or government agencies implement them. The cost and complexity of study designs that would adequately evaluate real-world structural interventions do not align well with the typical NIH-funded randomized control trial (RCT) model; this presumably could explain the dearth of research. Third, major challenges remain in evaluating the impact of structural interventions. Few NGO interventions are evaluated rigorously to rule out alternative explanations of effects. Perhaps most importantly, few NGO program evaluations involve a control group. Further, most structural intervention assessments are limited to either structural variables on which they directly intervene (such as social norms that condone intimate-partner violence or microcredit program use rates) or key HIV health outcomes only (Gupta et al. 2008). These research gaps in the development, implementation, and evaluation of structural interventions limit their wider dissemination and scale-up in resource-poor countries, where services are desperately needed.
This manuscript describes the rationale, design, and methodology of a five-year study, called SAGE4Health, examining a structural, multilevel intervention carried out by a large NGO, CARE International, in rural Malawi. This longitudinal study represents one of the first attempts to understand the mechanisms and processes through which changes in food security and economic outcomes (i.e., income, household assets, livelihood options) can impact HIV vulnerability (i.e., HIV risk behaviors, malnutrition, HIV infections). It also represents one of the first NIH-funded studies based on an academic-economic development NGO partnership (Delisle et al. 2005; Weinhardt et al. 2009). This type of partnership leverages strengths of NGOs (i.e., their ability to respond quickly to crises and their capacity for large-scale, sustainable development work) and HIV/AIDS researchers’ rigorous study designs and evaluations. In addition to examining pathways linking distal ecosocial factors to HIV vulnerability, this study will provide important information for understanding the impact of multilevel structural interventions on HIV with the potential for sustainable long-term public health benefits. Finally, this collaboration provides a unique opportunity to conduct a detailed study of a multilevel intervention on a scale unlikely to be supported entirely by NIH research funding; in effect, we use the NIH and NGO program funding to enhance both contributions.
Overview of the study design, and participants
Our study, “SAGE4Health”, a five-year project, comprises three interrelated samples designed to systematically evaluate “Support to Able-Bodied Vulnerable Groups to Achieve Food Security (SAFE),” a community-based, structural, multilevel health and development intervention. Details about SAFE are described later. In February 2009, enrollment of households for the SAGE4Health study began in rural areas of the Lilongwe district of central Malawi. The overall goal of the study is to elucidate pathways between socioeconomic environment, food security, HIV risk behaviors and HIV-relevant outcomes. Figure 1 displays the SAGE4Health project study design and sampling procedure and Figure 2 shows the study timeline.
The first sample uses a longitudinal, quasi-experimental, nonequivalent-control group design (Shadish et al. 2001). The objective is to examine the magnitude and mechanisms of the impact of SAFE intervention on economic status, food security, HIV/AIDS vulnerability and other health-related outcomes at the SAFE program participants’ level. Participants (n = 598) from three Traditional Authorities (TAs) who received SAFE intervention (intervention group) are compared with 301 participants who live in three other matched TAs (control group, matched on demographics and distance from an urban center) not receiving SAFE. Quantitative data was collected in three waves: Baseline (during year 2009), 18-month follow up and 36-month follow up.
The second sample is derived from a random sample community survey with a cohort-sequential design (Prinzie and Onghena 2005; Whitley and Kite 2012). The strength of the cohort-sequential design is that it combines cross-sectional and longitudinal approaches by starting a new cohort each time an assessment is made. The objective is to check the possible threats (i.e., other external factors such as Malawi's national fertilizer and seed programs that were introduced in the same period as the intervention) to the internal validity of the intervention/evaluation by examining whether the intervention effects were the result of something plausible during the study period in the larger communities where the intervention was delivered. To achieve this objective, we randomly selected villages in SAFE TAs and continued interviewing all households in these villages until we had a sample of 500 households that were not direct participants in the SAFE program. We followed the same procedure for the control TAs, where the SAFE program had not been implemented. We conducted the same assessments as in sample one during the same time frame at baseline and 36-month follow-up.
The third sample consists of a series of in-depth qualitative interviews and focus groups conducted 18 months after enrollment ― near the end of SAFE program implementation. The objective is to understand SAFE participants’ experiences in the program, their perceptions of its impact and their perspectives on the phenomena under study ― food security, economic livelihood and HIV vulnerability.
The study was approved by the following Institutional Review Boards (IRBs): (a) Medical College of Wisconsin Human Research Review Committee and (b) National Health Sciences Research Committee, Malawi Ministry of Health. Written informed consent for participation in the study was obtained from all participants, and by parents/guardians for the children under 5 for the anthropometric measurements.
The SAGE4Health research team consists of an interdisciplinary NGO-University partnership between CARE USA, CARE Malawi, the University of Wisconsin-Milwaukee (UWM) Joseph J. Zilber School of Public Health, the UWM College of Nursing’s Center for Global Health Equity, the Medical College of Wisconsin’s Center for AIDS Intervention Research, and the University of Malawi. The SAGE4Health study name reflects the key goals of the CARE’s SAFE Program: Savings, Agriculture, Governance and Empowerment. This partnership allowed the academic institutions and NGO to each contribute areas of expertise and learn from one another, rather than creating a costly situation whereby academia attempts to reinvent and implement an intervention in isolation. During the project, the academic collaborators provided the field staff, who had much experience in NGO-based field data collection but less experience in scientific research, with extensive in-person training in human subject research ethics, interviewing skills, data management, data analysis, scientific literature reviews, electronic data collection, participant tracking, oral presentation skills, and dissemination of results.
The study is being conducted in the rural areas of the Kasungu District of central Malawi. Among Malawian adults aged 15–49, approximately 11% live with HIV (National Statistical Office (NSO) and ICF Macro (2011)). Malawi’s population is young, rural, and poor National Statistical Office (NSO) and ICF Macro (2011) and 74% live below international poverty line of US$1.25 per day (UNICEF 2013). The Malawi economy is dominated by the agriculture sector, which employs 80% of the population, accounts for 42% of national GDP, supplies 81% of foreign exchange earnings and contributes significantly to national and household food security (Mataya 2003). Aside from agriculture, Malawi’s economy is also highly influenced by foreign aid. Based on the World Bank Africa Development Indicators 2011 Report, foreign aid accounted for 16.3% of Malawi’s GDP in 2009 (http://data.worldbank.org/sites/default/files/adi_2011-web.pdf). Given that Malawi’s economy receives substantial amounts of assistance, there is great interest from both donors and the Malawian government to understand the types of interventions that effectively create sustainable change in both the health and economic sectors.
Overview of the CARE Malawi SAFE Intervention
We are evaluating the program “Support to Able-Bodied Vulnerable Groups to Achieve Food Security (SAFE),” which was developed and implemented from January 2008-December 2010 by CARE-Malawi, a country office of CARE International, a large NGO focused on ending poverty globally. The SAFE intervention was designed to address closely intertwined structural issues contributing to HIV susceptibility and poor health: food insecurity, poverty, gender inequity, and ineffective governance. The ultimate objective was to produce potentially sustainable improvements in the country’s socioeconomic fabric by facilitating those at greatest risk to move up a ‘ladder’ of interventions and services toward economic security and decreased health risks. SAFE was implemented in three TAs (Njombwa, Kaomba, and Mwase)—geographic subdivisions of Kasungu District, located in west-central Malawi. It was funded primarily by the European Commission and partially by the Austrian Development Cooperation.
The SAFE intervention had four main components (Table 1) addressing four major issues: (1) improving farming practices and sustainable agriculture through Farmer Field Schools, (2) increasing access to savings and investment through Village Savings and Loans Groups (VSL), (3) building capacity of local governance structures, and (4) integrating HIV education and gender empowerment into programs through training and education. Each component utilized feasibility studies, relationship building, and extensive community sensitization prior to implementation. Overall, more than 9,000 households were educated by Farmer Field Schools, 443 VSL groups formed and hundreds of village leaders trained regarding gender issues (Chima 2011). Details of each component, including objectives, activities, and outcomes are presented in Table 1.
Below we delineate investigations using quantitative samples one and two, and the qualitative study, sample three. We discuss the assessment, power calculation (for quantitative studies only), and the data collection and analysis plan separately.
Primary outcome assessment for quantitative investigation one and investigation two
Quantitative studies that involved samples one and two use the same survey instrument consisting of 23 modules (Table 2). We selected those measurement modules because they correspond to constructs of a social-ecological framework (Krieger 1994; Loevinsohn and Gillespie 2003) of HIV vulnerability that aligns with the multilevel nature of the SAFE intervention (Figure 3). The scales assess major structural variables, including economic status and food security, HIV vulnerability outcomes, such as self-reported HIV testing, infections, and individuals’ HIV/AIDS risk behavior and risk perceptions. In addition, gender empowerment and gender-based violence are measured. Also included is a verbal autopsy section to record and determine the cause of death of family members. Anthropometric measurements are taken to quantitatively assess the nutritional status of study participants and children under 5 in the household.
Malawian field scientists administered the surveys face-to-face in Chichewa, the language spoken in the study area. Interview data were collected using Samsung Galaxy tablets utilizing the survey research platform Open Data Kit (Anokwa et al. 2009). The password-protected data were uploaded daily from the tablets to a desktop computer.
Major outcomes: Structural factors
Instruments measuring economic factors were adapted from the Malawi Diffusion and Ideational Change Project (MDICP) led by Watkins et al. (Anglewicz et al. 2009; Chin 2010) and previous CARE surveys. We focus selectively on important indicators of standard of living, consumption, production and income. Standard of living is measured by assets, income and consumption of basic commodities. Household production is measured by information on the quantities of major agricultural staples produced and the revenues of family enterprises in the past year. We also asked respondents about their relative wealth compared with other family members (specifically, siblings), and compared with other community members.
Food security, agricultural practices and nutrition outcomes
Food security is assessed with six questions from the preliminary CARE survey. These include questions about the months in the past year when the respondent’s household had enough food, projections for sufficient food crop production for the coming year and the household’s strategies when there was not enough food. Agriculture and natural resources are assessed with questions from CARE’s preliminary survey. For each of 23 sustainable agriculture practices (e.g., crop diversification, terracing, contour ridging, mulching), we ask respondents (1) if they know how to do it, (2) if they are applying it, and (3) if they are receiving benefit from it.
Nutrition is assessed with questions about the numbers of meals that household members ate in the past 24 hours, and 13 categories of foods consumed in the past 24 hours, with a yes/no response format. Anthropometric indicators including height, sitting height and weight are also collected for all respondents. Each measure is taken twice. If the difference is too large (e.g., > 1 cm for height), the measurement is repeated and an average of the two closest measures is used. BMI (Body Mass Index = weight/height2) and height-for-age (related to stunting) and weight-for-height (related to wasting) for children under the age of 5 are calculated, using the WHO/NCHS/CDC reference standards (de Onis et al. 2009, de Onis et al. 2006).
Major outcomes: HIV outcomes, HIV risk behavior and vulnerability variables
Subjective HIV testing, subjective HIV status and infection risk
Subjective HIV status and subjective HIV infection risks are elicited by having the respondents state the likelihood (from 0 = no risk to 10 = absolutely infected) that they (i) perceive themselves to be already infected, (ii) become infected in the next 5 years and (iii) ever become infected. These measures were developed for the MDICP (Anglewicz et al. 2009). HIV risk behavior (Weinhardt et al. 1999, 1998) measures (i.e., unprotected and protected sexual behaviors and sexually transmitted diseases) and a concise battery of measures derived from the Information-Motivation Behavior Skills Model (Fisher et al. 2002), as well as other relevant individual-level psychological factors, also were collected. HIV testing was assessed via self-report. Each participant was asked if they had ever been tested for HIV and, if so, what the result was.
This study’s statistical power is determined for the primary HIV sexual risk outcome and the primary food security outcome. The primary study hypothesis was that the intervention group was associated with a greater increase in HIV testing and a greater increase in food security than the control group at the 18- and 36-month follow-ups. The analysis was based on longitudinal trajectory analysis comparing conditions. The sample size was 598 for the control group and 301 for the intervention group. Effect sizes (d) were given according to Cohen (1988). Based on the generalized estimating equations (GEE) (Liang and Zeger 1986) and the power calculation by Lin and Myers (2006), using a significance level of α = 0.05, the power to detect a moderate difference (effect size = 0.5) is over 90%.
Data analysis for quantitative studies
Standard descriptive techniques are used to summarize and report study results. All data are inspected for skew and homogeneity of variance prior to data analysis. Data analysis of intervention outcomes is ongoing as of 2014. It is expected that distributions of some quantitative measures of sexual risk behavior data will be positively skewed, and transformations toward normality will be necessary. Non-parametric or generalized non-normality modeling techniques will be used for analyses if data are unable to be transformed to normal distributions. For hypothesis testing, the alpha level is set at 0.05.
Data analyses will utilize the full complexity (multilevel and repeated measures) of the data within and between individual households and between TAs. The strategy will use multiple analytical approaches: multivariate models, and generalized mixed linear models for longitudinal outcome data, coupled with structural equation modeling (SEM) of the pathways between primary outcomes and potential mediators suggested in an HIV ecosocial framework (Krieger 1994; Loevinsohn and Gillespie 2003).
This analysis plan is conceptualized as involving three phases: (1) repeated-measures analyses of HIV vulnerability, food security and economic outcomes assessed at baseline, and at 18- and 36-month intervals for participants in the SAFE participant- and non-SAFE comparison cohort; (2) analysis of HIV outcomes and vulnerability, food security and economic indicators from the random household community surveys in intervention and control TAs at baseline and at 36 months; (3) model-based mediation analysis for HIV vulnerability, food security and economic outcomes, based on the study cohort data. Furthermore, we will conduct gender moderator analysis. The purpose of gender moderator analysis is to examine whether the intervention has a different influence on men and women, and if so, why.
Qualitative study: sample three
The objective of Sample 3 data collection and analysis is to understand SAFE participants’ perceptions of the intervention’s impact, and their perspectives on the major phenomena under study ― food security, economic livelihood and HIV vulnerability. This third sample consists of 90 SAFE participants (60 women and 30 men) from intervention group households (N = 600) who, at the end of the SAFE Program, participated in focus groups, as well as in-depth qualitative interviews about their intervention experiences. We used stratified purposive sampling (Ritchie et al. 2014) to mirror the composition of Sample 1, to represent equally the three TAs in which the intervention took place, and to capture diversity of exposure to all elements of the intervention. The nine focus groups and 90 individual interviews allowed us to reach data saturation and redundancy.
At the end of SAFE program implementation by CARE, audio-recorded focus groups and individual in-depth interviews were conducted in Chichewa face-to-face by the Malawian field scientists. The timing of the qualitative data collection with Sample 3 corresponded to the timing of the 18-month follow-up with Sample 1 (Table 3). The purposive sample of 20 women and 10 men from each of the three TAs (N = 90) participated in a total of nine sex-segregated focus groups and 90 interviews. Questions posed to the focus groups asked participants to reflect on how SAFE impacted them at a community level ― What changes had they observed in the village and larger community around them as a result of the intervention? They were also asked to discuss collectively the food security, economic well-being, and HIV risk circumstances they encountered in their communities before, during and at the close of the SAFE Program. In the individual interviews, participants were asked to describe the effects of SAFE at a personal level ― what changes had they observed in their own households as a result of the intervention? How did they personally assess the food security, economic well-being and HIV risk circumstances they dealt with every day?
These qualitative data were transcribed, translated into English, and verified and discussed with the field scientists, according to quality assurance guidelines. Qualitative data analysis strategies (Spradley 1979) are being applied to examine what changes in their communities, and in their households participants attribute to the SAFE intervention, and how they perceive the intervention to have worked. We are describing their viewpoints on food security, livelihood strategies, economic stability and HIV vulnerability; and identifying in their accounts how these structural dynamics operate in women’s and men’s lives.
We are using thematic analysis techniques (Braun and Clarke 2006; Mkandawire-Valhmu and Stevens 2010; Schensul et al. 1999) to code discrete units of meaning, chart the relationships among these units and describe the patterns of experience seen in the focus group data. Individual interview data are being analyzed narratively (Riessman 2008; Stevens and Galvao 2007; Stevens and Hildebrandt 2009), attending to the personal stories participants tell about their own experiences in relation to: a) obtaining food on a daily basis, b) securing income, and c) attempting to manage HIV risk. For each participant, we are coding story content and context in a dialectical process, examining reported events as well as participants’ interpretations of what happened to them. We will follow this within-case analysis with an across-case analysis in which we search for similarities and differences among participants in the stories they tell about how SAFE affected their daily lives. Combining within-case and across-case approaches produces more contextually grounded, transferable findings (Ayers et al. 2003). To further support the authenticity of findings and auditability of analytic processes, we are a) engaging in inter-rater reliability activities as we create and apply codes, b) writing memos about analytic decisions, and c) validating findings with the field scientists.
Preliminary analyses consist of baseline comparisons between the intervention and comparison groups in samples 1 and 2. Independent t-tests were used for continuous data and chi-squared tests were used for categorical data. Table 4 displays baseline comparisons. Participants in the intervention group (sample 1) did not differ substantially from those in the control group in terms of sociodemographic traits. However, there was evidence at baseline that participants in the intervention group were older (p = .040), had smaller sizes of households (p = .001) and the households were less dominated by males (p = .039). In terms of primary outcomes, intervention group and control group had significant differences in income sources (p = .025), self-report “ever tested for HIV” (p = .001), as well as practice/applying sustainable agriculture methods. Community random samples (sample 2) in the intervention and control areas were much the same in their sociodemographic profiles, except on household size and marital status. Differences in selected outcomes between community random samples in intervention and control areas were observed in terms of cash income source, self-report “ever tested for HIV” and sustainable agriculture methods practice/application.
In linking academic research to a “real world,” NGO-led structural intervention, this study design may yield many benefits. First, the cost sharing of the implementation and the research between the NGO and the academic institution allowed for a fully-funded project that may not have been possible had only one funding stream been available. In an environment of constricting research funding, it is prudent to develop such mutually beneficial partnerships. Furthermore, the NGO-academia relationship extended beyond a grant application into capacity building. The NGO field staff developed expertise in research methods, including study protocols, human research ethics trainings, institutional review board (IRB) procedures, sampling methods, qualitative interview skills, data management techniques and computer-assisted quantitative interview skills; the academic staff gained insight into the logistics of on-the-ground program implementation, including international IRB processes, multi-agency grants management, logistics planning in challenging environments, NGO operations and interdisciplinary communication.
Due to the rapid pace with which NGOs undertake programs, compared to the relatively slow pace of federally funded academic research and result dissemination, this partnership has the potential to lead to rapid use of results. Given the program ownership by the NGO, the organization and its funders have a vested interest in understanding the effectiveness, or lack thereof, of its interventions. As the NGO and academic institution work in partnership, results are fed back to the implementers on a more real-time basis. The NGO, thus, has the opportunity to adjust future programming based on this study’s findings.
Limitations of this study include the use of self-reported outcomes and the quasi-experimental design. There is no random assignment to condition at the individual or community level. As such, the intervention and control groups may systematically differ in factors other than intervention exposure. As shown in Table 4, however, despite the nonequivalent-control group design, the intervention and control groups were very similar at baseline on key outcome variables. Another limitation of our design is that we cannot measure the separate effects of each intervention component. However, we feel the benefits of understanding the effects of the program as a whole, as designed, outweigh the benefit of a randomized treatment dismantling design. Bonell et al. (2006), Victora et al. (2004) and others have proposed designs such as ours to evaluate complex interventions in field studies.
There are many perspectives on whether structural interventions and public health should be evaluated using the RCT. Victora et al. argue that blind acceptance of RCT designs may not provide the entire picture of an intervention’s effects (Victora et al. 2004). They note, “Current trends toward acceptance of RCTs as the gold standard source of evidence may limit the knowledge base needed to make sound decisions about public health priorities, and policies” (Victora et al. 2004). External validity and generalizability are key to evaluations of the type presented here, as the partner NGO has the power to utilize findings to make programming changes, as well as replicate the intervention to other areas. RCTs are not the ideal study design when equipoise does not exist, where the intervention is already underway, and where tailoring of the intervention is not possible — for example, it must be consistent across an entire country as in some educational programs (Bonell et al. 2011). Though it would have been possible to construct an RCT of the SAFE intervention, it would not have fit into the established NGO mode of operation.
In addition to examining pathways linking environmental factors to HIV vulnerability, this research will yield important information for understanding the impact of multilevel environmental/structural interventions on HIV with the potential for sustainable long-term public health benefits. Finally, this collaboration represents a unique opportunity to conduct a rigorous effectiveness study of a multilevel intervention being implemented on a scale that is unlikely to be supported entirely by NIH research funding; in effect, we are leveraging the NIH and NGO program funding to enhance the contributions of both.
Acquired immune deficiency syndrome
Farmer Field Schools
Gross domestic product
Human Immunodeficiency Virus
Institutional Review Board
National Institutes of Health
Randomized controlled trials
Support to able bodied vulnerable groups to achieve food security
Savings, Agriculture, Governance & Empowerment for Health Study
University of Wisconsin Milwaukee
Village Savings and Loans.
Aarehag L, Dureval D, Sjöblom M, de Vlyder S: Country Economic Report. The impact of HIV/AIDS on the economy, livelihoods, and poverty of Malawi. Available via Swedish International Development Cooperation Agency (SIDA). 2006. . Accessed 1 Jan 2014 http://www.eldis.org/go/home%26id=51734%26type=Document#.U7GtP7Hqn-K
Anderson SA: Core indicators of nutritional state for difficult-to-sample populations. J Nutr 1990, 120(Suppl 11):1559-1600.
Anema A, Vogenthaler N, Frongillo E, Kadiyala S, Weiser S: Food insecurity and HIV/AIDS: current knowledge, gaps, and research priorities. Curr HIV/AIDS Rep 2009, 6(4):224-231.
Anema A, Chan K, Chen Y, Weiser S, Montaner JSG, Hogg RS: Relationship between food insecurity and mortality among HIV-positive injection drug users receiving antiretroviral therapy in British Columbia, Canada. PLoS One 2013, 8(5):e61277.
Anglewicz P, Adams J, Obare F, Kohler H-P, Watkins S: The Malawi diffusion and ideational change project 2004–06: data collection, data quality, and analysis of attrition. Demogr Res 2009, 20(21):503.
Anokwa Y, Hartung C, Brunette W, Borriello G, Lerer A: Open source data collection in the developing world. Computer 2009, 42(10):97-99.
Ayers L, Kavanaugh K, Knafl KA: Within-case and across-case approaches to qualitative data analysis. Qual Health Res 2003, 13(6):871-883.
Bonell C, Hargreaves J, Strange V, Pronyk P, Porter J: Should structural interventions be evaluated using RCTs? The case of HIV prevention. Soc Sci Med 2006, 63(5):1135-1142.
Bonell CP, Hargreaves J, Cousens S, Ross D, Hayes R, Petticrew M, Kirkwood BR: Alternatives to randomisation in the evaluation of public health interventions: design challenges and solutions. J Epidemiol Community Health 2011, 65(7):582-587.
Braun V, Clarke V: Using thematic analysis in psychology. Qual Res Psychol 2006, 3: 77-101.
Bryceson DF, Fonseca J, Kadzandira J: Social Pathways from the HIV/AIDS Deadlock of Disease, Denial and Desperation in Rural Malawi. Available via Southern African Regional Poverty Network (SARPN). 2004. . Accessed 1 Jan 2014 http://www.sarpn.org/documents/d0000906/index.php
Central Intelligence Agency (CIA): The World Factbook. 2010. Available via Central Intelligence Agency. . Accessed 1 Oct 2013 https://www.cia.gov/library/publications/the-world-factbook/rankorder/2155rank.html Available via Central Intelligence Agency. . Accessed 1 Oct 2013
Chima J: Support to able bodied vulnerable groups to achieve food security project (SAFE) (FOOD/2007/141–395) (2008–10): End of project evaluation final report. CARE Malawi/ICRISAT/CARE Austria/EU, Lilongwe; 2011.
Chin B: Income, health, and well-being in rural Malawi. Demogr Res 2010, 23(35):997-1030.
Cohen J: Statistical Power Analysis for the Behavioral Sciences. 2nd edition. Lawrence Erlbaum, New Jersey; 1988.
de Onis M, Onyango AW, Borghi E, Garza C, Yang H: Comparison of the World Health Organization (WHO) child growth standards and the National Center for Health Statistics/WHO international growth reference: implications for child health programmes. Public Health Nutr 2006, 9(07):942-947.
de Onis M, Garza C, Onyango AW, Rolland-Cachera MF: Les standards de croissance de l’organisation mondiale de la santé pour les nourrissons et les jeunes enfants. Arch Pediatr 2009, 16(1):47-53.
de Waal A, Whiteside A: New variant famine: AIDS and food crisis in Southern Africa. Lancet 2003, 362(9391):1234-1237.
Delisle H, Roberts J, Munro M, Jones L, Gyorkos T: The role of NGOs in global health research for development. Health Res Policy Syst 2005, 3(1):3.
Fisher JD, Fisher WA, Bryan AD, Misovich SJ: Information-motivation-behavioral skills model-based HIV risk behavior change intervention for inner-city high school youth. Health Psychol 2002, 21(2):177-186. doi:10.1037/0278-6220.127.116.11
Gupta GR, Parkhurst JO, Ogden JA, Aggleton P, Mahal A: Structural approaches to HIV prevention. Lancet 2008, 372(9640):764-775.
Hargreaves JR: Incorporating a Structural Approach Within Combination hiv Prevention: An Organising Framework. Paper Presented at the Structural Approaches to HIV Prevention Position Paper Series, Arlington, June 2013. 2013.
Integrated Household Survey (IHS3): National Statistical Office (NSO). The Commissioner of Statistics, Zomba; 2010-2011.
Johnson BT, Carey MP, Marsh KL, Levin KD, Scott-Sheldon LA: Interventions to reduce sexual risk for the human immunodeficiency virus in adolescents, 1985–2000: a research synthesis. Arch Pediatr Adolesc Med 2003, 157(4):381-388.
Kalichman SC, Cherry C, Amaral CM, Swetzes C, Eaton L, Macy R, Grebler T, Kalichman MO: Adherence to antiretroviral therapy and HIV transmission risks: implications for test-and-treat approaches to HIV prevention. AIDS Patient Care STDS 2010, 24(5):271-277.
Kim JC, Watts CH, Hargreaves JR, Ndhlovu LX, Phetla G, Morison LA, Busza J, Porter JDH, Pronyk P: Understanding the impact of a microfinance-based intervention on women’s empowerment and the reduction of intimate partner violence in South Africa. Am J Public Health 2007, 97(10):1794-1802.
Krieger N: Epidemiology and the web of causation: has anyone seen the spider? Soc Sci Med 1994, 39: 887-903.
Liang KY, Zeger SL: Longitudinal data analysis using generalized linear models. Biometrika 1986, 73: 13-22.
Lin HY, Myers L: Power and Type I error rates of goodness-of-fit statistics for binomial generalized estimating equations (GEE) models. Comput Stat Data Anal 2006, 50(12):3432-3448.
Loevinsohn ME, Gillespie S: HIV/AIDS, Food Security, and Rural Livelihoods: Understanding and Responding. Working Paper no. 2, IFPRI Discuss Pap Ser 157. 2003.
Maes KC, Hadley C, Tesfaye F, Shifferaw S: Food insecurity and mental health: surprising trends among community health volunteers in Addis Ababa, Ethiopia during the 2008 food crisis. Soc Sci Med 2010, 70(9):1450-1457.
Masanjala W: The poverty-HIV/AIDS nexus in Africa: a livelihood approach. Soc Sci Med 2007, 64(5):1032-1041.
Mataya C: Malawi. WHO Agreement on Agriculture: The Implementation Experience—Developing Country Case Studies. Available via: The Food and Agriculture Organization of the United Nations 2003. . Accessed 1 Oct. 2013 http://www.fao.org/docrep/005/y4632e/y4632e0n.htm#fn85
Miller C, Bangsberg D, Tuller D, Senkungu J, Kawuma A, Frongillo E, Weiser S: Food insecurity and sexual risk in an HIV epidemic community in Uganda. AIDS Behav 2011, 15: 1512-1519.
Mkandawire-Valhmu , Stevens PE: The critical value of focus group discussions in research with women living with HIV in Malawi. Qual Health Res 2010, 20: 684-696.
National Statistical Office (NSO) and ICF Macro: Malawi Demographic and Health Survey 2010. NSO and ICF Marco, Zomba and Calverton; 2011.
Nelson K, Brown ME, Lurie N: Hunger in an adult patient population. JAMA 1998, 279(15):1211-1214.
Normén L, Chan K, Braitstein P, Anema A, Bondy G, Montaner JSG, Hogg RS: Food insecurity and hunger are prevalent among HIV-positive individuals in British Columbia, Canada. J Nutr 2005, 135(4):820-825.
Olson CM: Nutrition and health outcomes associated with food insecurity and hunger. J Nutr 1999, 129(Suppl 2):S521-S524.
Prinzie P, Onghena P: Cohort Sequential Design. In Encyclopedia of Statistics in Behavioral Science. Edited by: Hoboken NJ. John Wiley & Sons, Ltd; 2005.
Pronyk PM, Hargreaves JR, Kim JC, Morison LA, Phetla G, Watts C, Busza J, Porter JD: Effect of a structural intervention for the prevention of intimate-partner violence and HIV in rural South Africa: a cluster randomised trial. Lancet 2006, 368: 1973-1983.
Riessman CK: Narrative Methods for the Human Sciences. Sage Publications, Inc., Thousand Oaks; 2008.
Ritchie J, Lewis J, Elam G, Tennant R, Rahim N: Designing and selecting samples. In Qualitative research practice: A guide for social science students and researchers. 2nd edition. Edited by: Ritchie J, Lewis J, Nicholls CM, Ormston R. Los Angeles: SAGE; 2014:111-145.
Schensul SL, Schensul JJ, LeCompte MD: Essential Ethnographic Methods. Altamira Press, Walnut Creek; 1999.
Shadish WR, Cook TD, Campbell DT: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. 2nd edition. Cengage Learning, Stamford; 2001.
Spradley JP: The Ethnographic Interview. Harcourt Brace Jovanovich College Publishers, Orlando; 1979.
Stevens PE, Galvao LW: “He won’t use condoms”: HIV-infected women’s struggles in primary relationships with serodiscordant partners. Am J Public Health 2007, 97(6):1015-1022.
Stevens PE, Hildebrandt E: Pill-taking from the perspective of HIV-infected women who are vulnerable to antiretroviral treatment failure. Qual Health Res 2009, 19(5):593-604.
Tsai AC, Bangsberg DR, Emenyonu N, Senkungu JK, Martin JN, Weiser SD: The social context of food insecurity among persons living with HIV/AIDS in rural Uganda. Soc Sci Med 2011, 73(12):1717-1724.
Tsai AC, Hung KJ, Weiser SD: Is food insecurity associated with HIV risk? Cross-sectional evidence from sexually active women in Brazil. PLoS One 2012, 9(4):e1001203.
UNICEF: Statistics. Malawi 2013. http://www.unicef.org/infobycountry/malawi_statistics.html Available via UNICEF. . Accessed 5 Nov 2013
Victora CG, Habicht J, Bryce J: Evidence-based public health: moving beyond randomized trials. Am J Public Health 2004, 94(3):400-405.
Wang EA, McGinnis KA, Fiellin DA, Goulet JL, Bryant K, Gibert CL, Leaf DA, Mattocks K, Sullivan LE, Vogenthaler N, Justice AC, VACS Project Team: Food insecurity is associated with poor virologic response among HIV-infected patients receiving antiretroviral medications. J Gen Intern Med 2011, 26(9):1012-1018. doi: 10.1007/s11606-011-1723-8
Weinhardt LS, Carey MP, Maisto SA, Carey KB, Cohen MM, Wickramasinghe SM: Reliability of the timeline follow-back sexual behavior interview. Ann Behav Med 1998, 20(1):25-30.
Weinhardt LS, Carey MP, Johnson BT, Bickham NL: Effects of HIV counseling and testing on sexual risk behavior: Meta-analysis of published research, 1985–1997. Am J Public Health 1999, 89(9):1397-1405.
Weinhardt LS, Galvao LW, Stevens PE, Masanjala WH, Bryant C, Ng’ombe T: Broadening research on microfinance and related strategies for HIV prevention: Commentary on Dworkin and Blankenship. AIDS Behav 2009, 13: 470-473.
Weiser S, Frongillo E, Ragland K, Hogg R, Riley E, Bangsberg D: Food insecurity is associated with incomplete HIV RNA suppression among homeless and marginally housed HIV-infected individuals in San Francisco. J Gen Intern Med 2009, 24(1):14-20.
Weiser SD, Fernandes K, Anema A, Brandson E, Lima VD, Montaner J, Hogg RS: Food Insecurity as a Barrier to Antiretroviral Therapy (ART) Adherence Among HIV-Infected Individuals. Paper Presented at the Fifth International AIDS Society Conference on HIV Pathogenesis, Treatment and Prevention, Cape Town, 19–22 July 2009. 2009.
Weiser SD, Tsai AC, Senkungu J, Emenyonu N, Tien P, Geng E, Hunt P, Martin J, Bangsberg DR: Food Insecurity and Morbidity Among HIV-Infected Persons Accessing Antiretroviral Therapy (ART) in Rural Uganda. Paper Presented at the VIII International AIDS Conference, Vienna, 18–23 July 2010. 2010.
Weiser SD, Young SL, Cohen CR, Kushel MB, Tsai AC, Tien PC, Hatcher AM, Frongillo EA, Bangsberg DR: Conceptual framework for understanding the bidirectional links between food insecurity and HIV/AIDS. Am J Clin Nutr 2011, 94(6):1729S-1739S.
Whitley BE Jr, Kite ME: Principles of Research in Behavioral Science. 3rd edition. Routledge, New York; 2012.
World Food Programme: Programming in the era of AIDS: WFP’s Response to HIV/AIDS. Key Publications 2003. Available via World Food Programme. . Accessed 2 Oct 2013 http://www.wfp.org/sites/default/files/Programming%20in%20the%20Era%20of%20AIDS%20WFP%27s%20Response%20to%20HIV:AIDS.pdf Available via World Food Programme. . Accessed 2 Oct 2013
We are grateful to all study participants from Malawi’s Kasungu District who kindly provided their time to participate in this study (written informed consent for participation in the study was obtained from all participants and by parents/guardians for the children under five years old for the anthropometric measurements); to CARE International-Malawi staff: Data Managers and Field Scientists, Gerald Kosamu and Esther Saka, and Field Scientists Chrissie Depete, Florence Kasende, Chrispine Manda, Thula Saka, Dziko Chatata, and Madalitso Yobe Goodson; Cecily Bryant, former Country Director for CARE International-Malawi; Francis Lwanda, former Assistant Country Director—Programs for CARE International-Malawi; Aldwin Mtembezeka, SAFE Project Manager, CARE International-Malawi; Madhu Deshmukh, Director of Special Initiatives, mHealth and Women’s Empowerment at CARE USA, Benjamin Schwartz, Senior Director for Health Programs at CARE USA and others who supported us. We also thank Melissa Lemke, Research Specialist at the Center for Urban Population Health, for support in project coordination and data management. Special thanks also to Anthony Barnett, Professor at the London School of Economics, for his advising as an Independent Consultant in the initial phases of the SAGE4Health Project. This study is supported by the National Institutes of Health, Eunice K. Shriver National Institute of Child Health and Human Development Grant R01-HD055868 (2008 – 2015).
The authors declare that they have no competing interests.
LW and LG conceptualized the study design detailed in the paper, led the team of co-authors and participated in the development of the paper, drafted portions of the paper, and reviewed and edited the article. KG wrote the initial draft and incorporated the edits, and reviewed the material that informs it. All authors participated in informing, reviewing and editing the paper and have approved the final version. LW had final responsibility for the decision to submit the paper for publication.
About this article
Cite this article
Weinhardt, L.S., Galvao, L.W., Mwenyekonde, T. et al. Methods and protocol of a mixed method quasi-experiment to evaluate the effects of a structural economic and food security intervention on HIV vulnerability in rural Malawi: The SAGE4Health Study. SpringerPlus 3, 296 (2014). https://doi.org/10.1186/2193-1801-3-296
- Food security
- Village savings and loans
- Quasi-experimental design