Skip to content

Advertisement

  • Research
  • Open Access

Development of a new multimedia instrument to measure cancer-specific quality of life in Portuguese-speaking patients with varying literacy skills

  • 1, 2, 3Email author,
  • 2, 4,
  • 2, 4,
  • 5,
  • 6,
  • 7,
  • 2,
  • 2, 4,
  • 8,
  • 9 and
  • 2, 3
SpringerPlus20165:972

https://doi.org/10.1186/s40064-016-2675-6

  • Received: 1 September 2015
  • Accepted: 26 June 2016
  • Published:

Abstract

Purpose

To develop and validate a new multimedia instrument to measure health-related quality of life (HRQOL) in Portuguese-speaking patients with cancer.

Methods

A mixed-methods study conducted in a large Brazilian Cancer Hospital. The instrument was developed along the following sequential phases: identification of HRQOL issues through qualitative content analysis of individual interviews, evaluation of the most important items according to the patients, review of the literature, evaluation by an expert committee, and pretesting. In sequence, an exploratory factor analysis was conducted (pilot testing, n = 149) to reduce the number of items and to define domains and scores. The psychometric properties of the IQualiV-OG-21 were measured in a large multicentre Brazilian study (n = 323). A software containing multimedia resources were developed to facilitate self-administration of IQualiV-OG-21; its feasibility and patients’ preferences (“paper and pencil” vs. software) were further tested (n = 54).

Results

An exploratory factor analysis reduced the 30-item instrument to 21 items. The IQualiV-OG-21 was divided into 6 domains: emotional, physical, existential, interpersonal relationships, functional and financial. The multicentre study confirmed that it was valid and reliable. The electronic multimedia instrument was easy to complete and acceptable to patients. Regarding preferences, 61.1 % of them preferred the electronic format in comparison with the paper and pencil format.

Conclusions

The IQualiV-OG-21 is a new valid and reliable multimedia HRQOL instrument that is well-understood, even by patients with low literacy skills, and can be answered quickly. It is a useful new tool that can be translated and tested in other cultures and languages.

Keywords

  • Quality of life
  • Development
  • Validation
  • Multimedia
  • Low literacy
  • IqualiV-OG-21

Background

The main advantage of measuring health-related quality of life (HRQOL) in oncology practice is the observation of the clinical benefit from the patient’s perspective. Most available patient-reported outcome (PRO) instruments were originally developed to be self-administered by individuals from developed countries, usually characterized by having higher levels of education than Brazilians. Of note, 75 % of Brazilians have difficulty reading and interpreting written texts (Ribeiro et al. 2002) and almost 80 % of Brazilian patients preferred that interviewers administer HRQOL assessment instruments (Brabo et al. 2006). While interviewer-administration is useful for respondents with reading difficulties, disadvantages include the costs required to hire, train and supervise interviewers, and the potential for interviewer bias (Selltiz et al. 1976). Other sources of bias include social desirability (the tendency to give a favorable picture of oneself) and acquiescent response sets (the tendency to agree/disagree with statements regardless of their content) (Edwards 1957; Crowne and Marlowe 1964).

A topic of interest is the content addressed by currently available PROs in oncology. Both most widely-used cancer-specific HRQOL instruments, i.e., the European Organization for the Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC QLQ-C30) and the Functional Assessment of Cancer Therapy-General (FACT-G), do not address existential issues. Specific modules for the evaluation of spirituality needs to be further supplemented, which increases the number of items and hence the time taken to complete the assessment tools. Spirituality, religiosity and personal beliefs seems to be of utmost importance for the coping strategies against cancer and its complications in Brazil, where only 8 % of individuals are not religious (IBGE 2012). Thus, we believe that a brief multidimensional cancer-specific PRO instrument that contains items addressing existential issues should be further developed.

Our hypothesis was that our PRO instrument should have been developed from the beginning focusing on the low educated patients to be used later by multimedia programs, and not a translated PRO instrument developed in other cultures and contexts. Several PROs have been developed for computer-based assessment. However, most of them were designed for use in a “paper and pencil” form and then adapted to the electronic format. The inclusion of multimedia resources (audio and/or visual aids) is a novel concept that can be especially useful for patients with low literacy skills. A new multimedia program—the “Talking Touchscreen”—was found to be acceptable, practical and user-friendly, thus providing opportunities to measure HRQOL in English- and Spanish-speaking patients with a range of literacy skills (Hahn et al. 2004; Yost et al. 2010).

The aim of this study was to develop and validate a new multimedia PRO instrument to measure HRQOL in Portuguese-speaking patients with cancer with varying literacy skills.

Methods

Ethics statement

The study complied with the ethical standards of the Declaration of Helsinki and Brazilian National Health Council resolution no. 466/2012. The study was approved by the Research Ethics Committee of the Barretos Cancer Hospital (Barretos-SP, Brazil, no 100.449). The Ethics Committee at the other participating centers also approved the study. Informed consent was obtained from all individual participants included in the study.

Phase I: Qualitative study—identification of the health-related quality of life content

Selection of participants

The inclusion criteria were as follows: histological diagnosis of cancer (regardless of the cancer type and current treatment), age 18 years or older, have knowledge about the diagnosis of cancer, and ability to communicate in the Brazilian Portuguese language. Patients with any uncontrolled psychiatric disease or significant cognitive dysfunction were not eligible to participate. In the Phase I, at least 5 patients should be of low education status (less than 8 years of formal education); two of them should be illiterate or with less than 2 years of education. Each subgroup (cancer survivors; systemic adjuvant or neoadjuvant treatment; systemic palliative treatment; palliative care only) should be composed of similar number of participants, in order to represent all stages of the cancer care continuum.

Data collection

A convenience sample of patients with cancer was recruited from the Outpatient and Inpatient Units of the Clinical Oncology and Palliative Care Departments. They were interviewed individually by two researchers with previous experience in conducting qualitative research (CEP and BSRP).

Patients answered the following eight open-ended questions: (1) How does your medical condition or your treatment interfere with your physical health?; (2) How does your medical condition or your treatment interfere with your emotional health?; (3) How does your medical condition or your treatment interfere with your social activities (going out, having fun, connecting with family and/or friends)?; (4) How does your medical condition or your treatment interfere with your finances?; (5) How does your medical condition or your treatment interfere with your religious/spiritual life?; (6) If you compare yourself today with when you were not sick, what do you think has changed in your life?; (7) What did you do before getting sick that you do not do today?; and (8) What worries you the most currently?

Data analyses

The interviews were recorded and transcribed verbatim for subsequent content analysis according to Bardin’s methodology (1994). The first step was the pre-analysis, which consisted of direct and intense contact with the material and organization of the data to meet the evaluation standards, including exhaustiveness, representativeness, homogeneity and relevance. The next step was to organize the topics according to their relevance and/or repetition (codification and categorization of the data). The transcripts were independently coded into categories by two researchers; disagreements in coding were resolved during a consensus meeting. Finally, the raw data were divided into percentages and interpreted (treatment and interpretation of the obtained results). The criterion of data saturation was used to define the sample size for this phase (Miles and Huberman 1994).

Phase II: Identification of the most relevant items

Instrument development strategy

Considering the content identified in the qualitative analysis, the authors created a pool of items for each category identified in the qualitative analysis. The items were grouped into a questionnaire using 5-point Likert-type responses (“not at all important”, “slightly important”, “moderately important”, “very important” and “extremely important”).

Selection of participants

The inclusion criteria were as follows: histological diagnosis of cancer (regardless of the cancer type and current treatment), age 18 years or older, have knowledge about the diagnosis of cancer, and ability to communicate in the Brazilian Portuguese language. Patients with any uncontrolled psychiatric disease or significant cognitive dysfunction were not eligible to participate.

Data collection

The instrument was administered to a sample of patients on “paper and pencil” format in private rooms at the outpatient units. Trained research coordinators from the Center for Research Support (Barretos Cancer Hospital) conducted all the interviews.

Data analyses

The items with the highest percentage of “extremely important” responses and those with the highest percentage of classification as among the 10 most important items were identified; these items were retained for the next study phase.

Phase III: Development of the instrument, pretesting and pilot testing

Instrument development strategy

The new instrument was developed with standardized questions that always followed the same format (starting with “how often…”) and used the same response scale (“never” to “very often”). A consensus indicated that the assessment period would be the 2 weeks immediately preceding the administration of the instrument, which was considered appropriate for patients with different stages of cancer. Considering that the ultimate goal was the development of software with audiovisual resources, an answer scale with figures was developed to facilitate the responses of patients with lower educational levels. The figures were built as squares that were filled with increasing amounts of small black spots and ranged from empty (never) to almost completely filled (very often) (see Additional file 1).

The instrument was evaluated by an expert committee that consisted of 8 members (3 clinical oncologists, 1 biologist, 3 nurses and 1 surgical oncologist), 3 of whom had experience developing and validating healthcare assessment instruments. All members independently critiqued the instrument using a form designed for the study. Two authors grouped all the reviews into a single, blinded version of the critiques that was subsequently evaluated by all committee members.

Selection of participants

The inclusion criteria were as follows: histological diagnosis of cancer (regardless of the cancer type and current treatment), age 18 years or older, have knowledge about the diagnosis of cancer, and ability to communicate in the Brazilian Portuguese language. Patients with any uncontrolled psychiatric disease or significant cognitive dysfunction were not eligible to participate. Pretesting was performed with 15 cancer patients who had different educational levels; at least 10 with low education level (less than 8 years of education) and among them, at least 5 with less than 2 years of education.

Data collection

In the pretesting, cognitive debriefing interviews were conducted by experienced interviewers using a semi-structured interview guide.

A subsequent pilot testing was performed with 149 patients with cancer recruited at the Barretos Cancer Hospital. All participants responded to the 30-item instrument in “paper and pencil” format.

Data analysis

Pretesting—Participants were requested to explain what they understood about each item, thus creating an assessment of the “participant’s comprehension of the item”. In addition, the patients noted whether the item was confusing or embarrassing and whether they suggested any change to the instrument. A think-aloud method with cognitive debriefing was used.

Pilot testing (reduction of items)—The instrument containing 30 items was initially submitted to an exploratory factor analysis. It was performed using principal component analysis and varimax rotation. An Eigenvalue >1 was defined to extract the factors in the exploratory phase. The adequacy of the data for the factor analysis was identified using the Kaiser–Meyer–Olkin (KMO) test and Bartlett’s test of sphericity. During the initial exploratory factor analysis, the items that had low factor loadings (<0.4) in all domains were excluded. Other exclusion criteria were a percentage of unanswered items >4 % and a correlation coefficient between items of the same domain >0.8. After the initial exclusion of items, a new exploratory factor analysis was performed to define the domains. Statistical analyses were performed using SPSS software (version 20.0, Chicago, IL, USA).

Phase IV: Evaluation of the psychometric properties of the IQualiV-OG-21—multicentre Brazilian study

Selection of participants

The inclusion criteria were as follows: histological diagnosis of cancer (regardless of the cancer type and current treatment), age 18 years or older, have knowledge about the diagnosis of cancer, and ability to communicate in the Brazilian Portuguese language. Patients with any uncontrolled psychiatric disease or significant cognitive dysfunction were not eligible to participate. Patients were recruited from four different Hospitals: Barretos Cancer Hospital (Barretos-SP, Brazil), Botucatu’s Medical School (Botucatu-SP Brazil), Oncology Institute of Ribeirão Preto (Ribeirão Preto-SP, Brazil) and Mater Dei Hospital (Belo Horizonte-MG, Brazil). The first two centers belong to the Brazilian public health system, and the other two serve insured or private patients.

Data collection

All participants responded the IQualiV-OG-21 in “paper and pencil” format and then completed both the Functional Assessment of Cancer Therapy-General (FACT-G) and the Functional Assessment of Chronic Illness Therapy-Spiritual Well-Being scale (FACIT-sp-12). To ensure high data quality, study procedures were standardized through in-person or video conference meetings.

Validation instruments

The Functional Assessment of Cancer Therapy-General (FACT-G) contains 27 items on a 4-point Likert-type scale assessing four domains of “well-being” (physical, social/family, emotional and functional); higher scores indicate better HRQOL (Cella et al. 1993). It has been translated to and validated for Portuguese (Ishikawa et al. 2010).

The Functional Assessment of Chronic Illness Therapy-Spiritual Well-Being scale (FACIT-sp-12) consists of 12 items and two sub-domains of spiritual well-being (peace/meaning and faith). Higher scores indicate higher levels of spiritual well-being (Peterman et al. 2002). It has been previously validated in Brazil (Lucchetti et al. 2013).

Data analyses

The factor structure for the IQualiV-OG-21, which was defined in Phase III, was tested in Phase IV (multicenter study); maximum likelihood estimation was used for the confirmatory factor analysis. To test goodness of fit we used the comparative fit index (CFI) and the Tucker-Lewis Index (TLI). For both, values ≥0.95 indicate a good fit, and values >0.90 an acceptable fit (Hu and Bentler 1999). Additionally, Root Mean Square Error of Approximation (RMSEA) values below 0.08 are considered to reflect acceptable fit to the model and values smaller than 0.05 as good fit (Schermelleh-Engel et al. 2003). As a requirement, all standardized factor loadings need to be greater than 0.4 and statistically significant.

The internal consistency reliability of each subscale was assessed using Cronbach’s coefficient alpha; values above 0.70 were considered adequate (Terwee et al. 2007).

Sixty four patients from the phase IV cohort were retested after 7–14 days to measure the test–retest reliability. It was assessed using the intraclass correlation coefficient (ICC); values above 0.70 were considered adequate (Terwee et al. 2007).

Known-group validity was assessed by comparing the mean IQualiV-OG-21 subscales scores between groups of patients with Eastern Cooperative Oncology Group Performance Status (ECOG-PS) values of 0–1 versus 2–4. A t test was used for these comparisons. Additionally, Cohen’s effect size (ES) was calculated to indicate the magnitude of the differences between groups, defined as the difference in scores between the ECOG-PS 0–1 and 2–4 groups divided by the combined standard deviation of both groups. Effect size values of <0.2, 0.2–0.49, 0.5–0.79 and ≥0.8 were classified as negligible, small, moderate and large differences, respectively (Fayers and Machin 2007). We hypothesized that the physical, functional and overall domains would be the most discriminative. For convergent validity, the IQualiV-OG-21 scores were compared with the corresponding scores on the FACT-G and FACIT-sp-12 instruments. Correlation coefficients ≥0.4 were considered adequate (Fayers and Machin 2007).

Statistical analyses were performed using SPSS software (version 20.0, Chicago, IL, USA) and confirmatory factor analysis was accomplished with SPSS Amos version. 20.0 (version 20.0, Chicago, IL, USA). A nominal significance level (p < 0.05) was used to interpret results.

Phase V: Development of the software containing the IQualiV-OG-21

Software development

In brief, the software was developed in Java programming language and contained audio–visual resources (recorded reading of the instructions, questions and response choices, and interactive icons). The software was designed so that even illiterate respondents could answer the instrument without the need for application by an interviewer. One item at a time is presented on the computer touchscreen. Small picture icons appear below each text element allowing patients to replay the sound if necessary, to skip to the next item without answering, and allowing return to previous items to correct errors or change answers.

Phase VI: Comparison between “paper and pencil” and software IQualiV-OG-21 formats

Selection of participants

The inclusion criteria were as follows: histological diagnosis of cancer (regardless of the cancer type and current treatment), age 18 years or older, have knowledge about the diagnosis of cancer, and ability to communicate in the Brazilian Portuguese language. Patients with any uncontrolled psychiatric disease or significant cognitive dysfunction were not eligible to participate.

Data collection

A new sample of fifty four cancer patients answered the IQualiV-OG-21 in both “paper and pencil” and software formats. The order of administration was randomized. Between the administrations of the instruments, patients were evaluated regarding their functional literacy levels using the Multidimensional Screener of Functional Health Literacy (MSFHL). The MSFHL is a simple screening tool which provides an accurate prediction of a patient’s functional literacy status. It consists of six items that provide a score ranging from 0 to 10; scores are categorized as inadequate (0–3), marginal (4–5) and adequate (6–10) functional literacy levels (Apolinario et al. 2014). In addition, patients reported their preferences about the best format (“paper and pencil” vs. indifferent vs. software). Afterwards, the patients were classified by the interviewer regarding their ability to answer the instrument in a self-administered format.

Data analyses

Descriptive statistics were used to describe differences between patient’s preferences and abilities to answer the instruments.

Results

Phase I: Qualitative study

Data saturation was identified after the initial evaluation of 20 patients. Table 1 shows the characteristics of the patients included in this phase. The interviews lasted approximately 20 min. After the transcriptions were coded, 5 categories were identified (“physical”, “emotional”, “financial”, “social” and “existential”). The category “physical” was subdivided into 7 subcategories; the category “emotional” into 4 subcategories; the category “financial” into 2 subcategories; the category “social” into 5 subcategories; and the category “existential” into 6 subcategories. Based on these results, the investigators developed a questionnaire with 43 items to use in the next study phase.
Table 1

Sociodemographic and clinical characteristics of the patients included in the study

Characteristic

Phase I

Phase II

Phase III

Phase IV

Phase VI

(n = 20)

(n = 60)

(n = 149)

(n = 323)

(n = 54)

n

%

n

%

n

%

n

%

n

%

Gender

 Male

8

40.0

27

45.0

49

32.9

151

46.7

36

60.0

 Female

12

60.0

33

55.0

100

67.1

172

53.3

24

40.0

Age (years)

 <40

1

5.0

12

20.0

39

26.2

45

13.9

10

16.7

 40–65

16

80.0

35

58.3

91

61.1

197

61.0

41

68.3

 ≥65

3

15.0

13

21.7

19

12.8

81

25.1

9

15.0

Educational level (years of formal schooling)

 Illiterate

0

0

0

0

4

2.7

16

5.0

4

6.7

 <8

5

25.0

23

38.3

56

37.6

145

44.9

27

45.0

 8–11

6

30.0

16

26.7

26

17.4

84

26.0

24

44.0

 >11

9

45.0

21

35.0

63

42.3

78

24.1

5

8.3

Family income (expressed as multiples of the minimum wage)

 ≤1

3

15.0

14

25.0

29

19.7

52

17.9

23

38.3

 1–2

8

40.0

11

19.6

51

34.7

63

21.7

18

30.0

 2.1–3

4

20.0

12

21.4

26

17.7

49

16.9

10

16.7

 >3

5

25.0

19

33.9

41

27.9

126

43.4

7

12.0

 Missing

0

 

4

 

2

 

33

 

2

 

Brazilian region of origin

 Southeast

14

70.0

42

70.0

106

71.1

257

79.6

43

71.7

 Southern

0

0

1

1.7

1

0.7

1

0.3

0

0.0

 Northern

1

5.0

4

6.7

7

4.7

22

6.8

7

11.7

 Midwest

5

25.0

13

21.7

34

22.8

40

12.4

8

13.3

 Northeast

0

0

0

0

1

0.7

3

0.9

2

3.3

Type of primary tumor

 Breast

5

25.0

12

20.0

53

35.6

73

22.6

5

8.3

 Colorectal

4

20.0

15

25.0

25

16.8

38

11.8

3

5.0

 Prostate

2

10.0

7

11.7

10

6.7

33

10.2

2

3.3

 Cervix

1

5.0

3

5.0

16

10.7

19

5.9

0

0

 Gastric

1

5.0

1

1.7

8

5.4

9

2.8

0

0

 Lung

0

0

4

6.7

3

2.0

22

6.8

0

0

 Head and Neck

0

0

4

6.7

2

1.3

33

10.2

40

66.7

 Esophageal

1

5.0

1

1.7

4

2.7

7

2.2

0

0

 Endometrial

0

0

0

0

2

1.3

1

0.3

0

0

 Others

6

30.0

13

21.7

26

17.4

88

27.2

10

16.7

Distant metastasis

 Yes

11

55.0

31

52.0

61

41

158

48.9

15

25.0

 No

9

45.0

29

48.0

88

59

165

51.1

45

75.0

Type of current treatment

 Follow-up (NED)

0

0

11

18.3

16

10.7

45

13.9

3

5.0

 Curative/Adj/Neoadj systemic therapy

4

20.0

18

30.0

72

48.3

120

37.2

45

75.0

 Palliative systemic therapy

10

50.0

21

35.0

56

37.6

132

40.9

10

16.7

 Palliative care only

6

30.0

10

16.7

5

3.5

26

8.0

2

3.3

Place of interview

 Ambulatory

9

45.0

56

93.0

117

78.5

288

89.2

59

98.3

 Hospital

11

55.0

4

7.0

32

21.5

35

10.8

1

1.7

ECOG performance status

 0–1

8

40.0

NA

109

73.2

220

68.8

54

90.0

 2–4

12

60.0

NA

39

26.2

100

31.3

6

10.0

 Missing

      

3

   

NED no evidence of disease, Adj Adjuvant, Neoadj Neoadjuvant, ECOG Eastern Cooperative Oncology Group

Phase II: Identification of the most relevant items

Sixty patients answered a questionnaire containing 43 items. The items with the highest percentage of “extremely important” responses and the percentage of times they were classified among the 10 most important were identified. From this analysis, the selected items were divided into physical symptoms (8 items), emotional problems (4 items), functionality (1 item), social problems (2 items) and existential problems (2 items), for a total of 17 items. After reviewing the literature, the authors of the instrument decided to include 8 more items that aimed to assess the following: functionality (1 item), anxiety (1 item), interpersonal relationships (2 items), fun and leisure (1 item) inner peace (1 item), irritability (1 item) and financial aspects (1 item). Table 1 shows the characteristics of the patients included in this phase.

Phase III: Construction of the initial version of the instrument, pretesting and pilot testing

The initial version of the instrument with 25 items was evaluated for content validity by an expert committee. Two authors of the study analyzed the committee members’ critiques and decided to include 5 more items to assess the following constructs: overall assessment of HRQOL (1 item), sexuality (1 item), meaning of life (1 item), prayer (1 item) and the importance of faith in fighting the disease (1 item). Additional file 2 details the construction of the initial version of IQualiV-OG with 30 items. The new version with 30 items was again evaluated by all members of the expert committee, who considered the content adequate for an instrument that aims to measure the HRQOL of patients with general cancer. Some adjustments were made to the instrument’s statements, the presentation order of a few items and the language of 3 items that were considered confusing by patients.

A total of 180 patients were invited to participate in the pilot test; of these, 152 agreed to participate. Three cases were subsequently excluded from the statistical analyses because of screening failure (there were cases of in situ carcinomas). Table 1 shows the sociodemographic and clinical characteristics of the study participants.

The KMO and Bartlett’s test values were adequate (KMO = 0.784, p < 0.001), indicating that the sample could be subjected to factor analysis. The exploratory factor analysis of the instrument with 30 items identified 9 factors that explained 68.5 % of the total variance. Nine items were excluded because they were grouped into factors other than the one they had been originally devised for. Table 2 shows the factor loadings for the 21 items that were retained. Item 1, which assesses the frequency of pain, was grouped into the emotional factor and not into the physical factor; however, considering this item’s importance to the instrument, the authors decided to keep it in the analysis of the domain for which it was devised (the physical domain). None of the items were excluded based on the percentage of missing responses or inter-item correlations within the same factor >0.8. After 9 items were excluded, the instrument was named the “Instrumento para avaliação da Qualidade de Vida em Oncologia Geral com 21 itens” (IQualiV-OG-21) and was composed of 21 items divided into 6 domains: emotional (6 items), physical (5 items), existential (3 items), interpersonal relationships (2 items), functional (3 items) and financial (2 items). Physical domain included items assessing physical symptoms (pain, fatigue, nausea, etc.); otherwise, functional domain included items measuring physical functioning (ability to work, take care of itself, and need to rest). The original instrument in Portuguese and a preliminary English translation are available in Additional file 1. The scores for each domain were developed using the mean of the answered items on a normalized scale ranging from 0 to 100, with higher scores representing worse HRQOL. We considered HRQOL as a multi-dimensional concept that includes different domains related each other; additionally, a composite score would represent the global HRQOL. Thus, the overall score, which is the mean of all domains, was also created.
Table 2

Factor analysis of the IQualiV-OG-21 (n = 149)

Item

Descriptorsa

Factors

Emotion

Physical

Existen

Relation

Function

Financial

6

Insomnia

0.482

0.064

0.129

0.280

0.126

0.281

17

Depressed

0.652

0.133

0.247

0.252

0.069

0.239

18

Anxious

0.736

−0.047

0.147

0.138

−0.064

0.251

19

Easily irritated

0.702

0.228

0.172

0.257

−0.095

0.023

21

Fear of worsening health

0.588

0.012

0.335

−0.207

0.160

0.145

22

Worry about the family’s future

0.572

0.032

0.055

−0.059

0.191

0.460

1

Pain

0.588

0.189

−0.165

0.358

0.348

−0.050

2

Weak or without energy

0.399

0.582

−0.045

0.197

0.219

−0.204

4

Lack of appetite

0.076

0.742

0.084

−0.089

0.118

0.347

5

Limited ability to taste foods

−0.061

0.755

0.095

0.153

0.161

0.081

7

Nauseated

0.142

0.777

0.117

0.062

0.009

0.121

24

Hopeless

0.203

0.111

0.784

0.073

0.169

0.008

25

Lost faith in God

0.144

0.068

0.786

0.215

−0.004

0.153

29

Life did not make sense

0.109

0.088

0.759

0.077

0.051

−0.009

15

Relationship problems

0.225

0.112

0.132

0.762

0.015

0.177

16

People distanced themselves

0.151

0.059

0.205

0.795

0.107

0.064

9

Needing help to get dressed, shower or eat

0.067

−0.001

0.020

−0.034

0.821

−0.017

10

Unable to work

0.107

0.260

0.120

0.313

0.689

0.318

11

Needing to lie down or sit to rest

0.058

0.407

0.243

0.043

0.648

0.021

12

Worried about finances

0.263

0.180

0.084

0.051

0.113

0.773

13

Not enough money to meet needs

0.259

0.180

0.017

0.298

−0.049

0.744

aThe brief item descriptors approximate the questions asked in each item

Factor loadings are italicized when grouped in a same domain

Emotion. emotional, Existen. existential, Relation. interpersonal relationships, Function. functional

Phase IV: Evaluation of the psychometric properties (multicentre study)

From the 379 patients invited to participate in the study, 323 agreed to participate; Barretos Cancer Hospital (n = 181), Botucatu’s Medical School (n = 49), Oncology Institute of Ribeirão Preto (n = 50) and Mater Dei Hospital (n = 43). Table 1 details the patient characteristics.

The IQualiV-OG-21 was completed in a mean (SD) of 6.0 (3.3) minutes, ranging from 1.2 to 23 min. None of the items had an unanswered percentage >4 %, values ranged from 0 to 0.9 %.

Regarding the confirmatory factor analysis, fit indices were considered good (CFI = 0.950, TLI = 0.933, RMSEA = 0.051) and all factor loadings were greater than 0.40 and statistically significant.

The domains with the lowest (most favorable) mean scores were the personal relationship (mean = 13.4) and existential domains (mean = 15.2), and the domains with the highest (least favorable) scores were the emotional (mean = 41.7) and financial (mean = 39.6) domains (Table 3). Internal consistency reliability was considered adequate, with Cronbach’s alpha values ranging from 0.75 (for the personal relationship domain) to 0.90 (for the overall score; Table 3).
Table 3

Mean values, internal consistency and test–retest analysis of IQualiV–OG-21

IQualiV-OG-21 domains

Number of items

Mean (SD)

Cronbach’s α (95 % CI)

ICC (95 % CI)a

Emotional

6

41.7 (24.9)

0.85 (0.82–0.87)

0.88 (0.80–0.93)

Physical

5

33.0 (23.8)

0.79 (0.75–0.82)

0.79 (0.66–0.88)

Financial

2

39.6 (32.4)

0.81 (0.77–0.85)

0.83 (0.72–0.90)

Existential

3

15.2 (22.2)

0.81 (0.77–0.84)

0.96 (0.94–0.98)

Functional

3

28.9 (25.9)

0.77 (0.72–0.81)

0.86 (0.77–0.92)

Personal relationships

2

13.4 (20.7)

0.75 (0.69–0.80)

0.94 (0.90–0.97)

Total score

21

28.7 (17.7)

0.90 (0.89–0.92)

0.95 (0.92–0.97)

SD standard deviation, ICC intraclass correlation coefficient

an = 64

Test–retest reliability was assessed in 64 participants using the ICC and was considered adequate in all domains, ranging from 0.79 (physical domain) to 0.96 (existential domain; Table 3).

Convergent validity was confirmed by the Pearson correlation between the IQualiV-OG-21 domains and the corresponding domains of the FACT-G and FACIT-sp-12 instruments. All expected correlations had coefficients greater than 0.4, with only two exceptions: personal relationship and FACT-G social (r = −0.242), and existential and FACIT-sp 12 faith (r = −0.315) (Table 4).
Table 4

Pearson correlations between IQualiV-OG-21 and FACT-G and FACIT-sp-12 (n = 323)

IQualiV-OG-21

FACT-G

FACIT-sp-12

Physical

Emotional

Functional

Social

Total

Meaning/peace

Faith

Spirituality total

Emotional

−0.55*

−0.63*

−0.40*

−0.07

−0.53*

−0.29*

−0.01

−0.24*

Physical

−0.72*

−0.47*

−0.47*

−0.17*

−0.61*

−0.41*

−0.25*

−0.38*

Financial

−0.33*

−0.31*

−0.27*

−0.09

−0.33*

−0.21*

−0.06

−0.17*

Existential

−0.51*

−0.60*

−0.48*

−0.18*

−0.57*

−0.48*

−0.32*

−0.46*

Functional

−0.70*

−0.45*

−0.61*

−0.21*

−0.65*

−0.44*

−0.28*

−0.41*

Personal relationships

−0.22*

−0.26*

−0.19*

−0.24*

−0.29*

−0.29*

−0.15*

−0.26*

Total score

−0.71*

−0.63*

−0.57*

−0.22*

−0.69*

−0.49*

−0.26*

−0.44*

* p < 0.01.  Underline indicates a pair of scales that should correlate theoretically

Known-groups validity was assessed by comparing the IQualiV-OG-21 scores of patients who presented better (ECOG-PS of 0-1) and worse (ECOG-PS of 2-4) functional performances. As expected, the magnitude of the difference was greatest in the following domains: physical (ES = 0.95), functional (ES = 1.16) and overall score (ES = 0.90; Table 5).
Table 5

Known-group validation according to the performance status

IQualiV-OG-21 domains

ECOG performance status

P a

ES

Mean (SD)

0–1

2–4

(n = 220)

(n = 100)

Emotional

37.6 (24.7)

50.9 (23.3)

<0.001

13.22

0.53

Physical

25.9 (22.0)

48.6 (20.2)

<0.001

22.71

0.95

Financial

33.7 (31.8)

51.6 (30.5)

<0.001

17.87

0.55

Existential

11.6 (20.6)

23.4 (23.5)

<0.001

11.87

0.53

Functional

19.6 (20.4)

49.8 (24.7)

<0.001

30.27

1.16

Personal relationships

13.5 (20.8)

13.4 (20.8)

0.952

−0.14

−0.00

Total score

23.6 (16.3)

39.6 (15.6)

<0.001

17.67

0.90

SD standard deviation, difference between the two groups, ES effect size

a T test

Phase VI: Comparison between “paper and pencil” and software IQualiV-OG-21 formats

Among the 54 participants in the method comparison sub-study, there were 29 (53.7 %), 9 (16.7 %) and 16 (29.6 %) patients classified as inadequate, marginal and adequate functional literacy levels, respectively, according to the Multidimensional Screener of Functional Health Literacy. Twelve (n = 12, 22.2 %; inadequate functional literacy = 10, marginal functional literacy = 2) participants required the application of the IQualiV-OG-21 in “paper and pencil” form by the interviewer. Interestingly, fifteen (n = 15, 27.8 %; inadequate functional literacy = 9, marginal functional literacy = 3, adequate functional literacy = 3) participants initially tried to self-administer the instrument in “paper and pencil” form, nevertheless they were unable to complete it; the application by the interviewer were also required. All participants were able to self-administer the electronic IQualiV-OG-21 and only 4 (7.4 %) patients required significant help by the interviewer. Regarding their preferences about the best format, 6 (11.1 %) patients preferred the “paper and pencil” format, 15 (27.8 %) reported both were equal, and 33 (61.1 %) preferred the electronic format.

Discussion

Principal findings

In this study, we present the first cancer-specific HRQOL instrument developed as a multimedia electronic tool for Portuguese-speaking patients. The IQualiV-OG-21 demonstrated good psychometric properties, and its electronic form was practical and well accepted by patients with cancer. By overcoming barriers to self-administration, this new instrument provides greater opportunities to measure HRQOL in patients with a range of literacy skills. Our findings are in accordance with previous studies conducted in patients with low literacy skills from Spanish- and Chinese-speaking populations (Hahn et al. 2003, 2004, 2007; Hahn and Cella 2003; Thumboo et al. 2006).

Limitations and strengths of the study

The study has some limitations. We enrolled only ambulatory cancer patients who were well enough to participate. Results may not be generalizable to patients with greater disease severity and poorer HRQOL. The personal relationship domain of the IQualiV-OG-21 was weakly associated with the social domain of the FACT-G. This may be due to the fact that the social domain of the FACT-G measures a broader concept than our tool which is focused on relationship quality. Future studies should correlate the personal relationship domain with other instruments that more closely measure similar concepts. Similarly, the lack of a standard instrument that can be used to correlate the financial domain scores of the IQualiV-OG-21 is another limitation of that could be addressed in future studies. It might be useful to consider other psychometric approaches to scoring, such as item response theory (Van der Linden and Hambleton 1997) or the Rasch model (Bond and Fox 2007). Another limitation is that the IQualiV was not developed and tested initially as software format, but in “paper and pencil” form. The software was developed throughout the initial phases of the study, therefore, unfortunately, it was considered ready for use only in the Phase VI. Further data are needed about the software in a large cohort of patients with cancer. Moreover, the comparison between IQualiV-OG-21 software and other standard similar questionnaires is awaited.

In terms of strengths, we can point out the originality of the study, since only a handful of multimedia-containing HRQOL instruments have been developed so far. Another strength is the external validation in a multicentre cohort comprising patients from different social and educational skills. Furthermore, the IQualiV-OG-21 includes a specific existential domain, which seems to be relevant for its use when evaluating patients with cancer, since spirituality and religiosity are commonly used as coping strategies by Brazilians (Paiva et al. 2011). The widely-used PRO instruments EORTC QLQ-C30 and FACT-G do not evaluate existential aspects and thus require the administration of specific supplements for this purpose.

Implications for clinicians and researchers

Brazil is a continental country, with about 200 million inhabitants and 576 thousand new cases of cancer each year (INCA 2014; IBGE 2015). HRQOL measurement has practical implications for use in cancer care, such as improving communication, screening symptoms and triggering protocols, monitoring responses, facilitating shared decision making, etc. (Hearn and Higginson 1997; Higginson and Carr 2001; Greenhalgh 2009). In addition, considering the large number of Brazilian patients with low literacy skills, the electronic IQualiV-OG-21 is a useful new instrument for routine practice and research. Responses are automatically recorded in spreadsheets and there will be cost saving because of the lack of need of typists and trained interviewers. Currently, we are leading the ProQualiV program (Programa de Qualidade de Vida como um Indicador Vital) aiming to establish the measurement of HRQOL as a routine vital indicator in a large comprehensive cancer hospital in Brazil; electronic IQualiV-OG-21 is an ideal tool to be used in this program.

Unanswered questions and future research

An unanswered question is the lack of assessment of the IQualiV-OG-21’s responsiveness, which is considered important for its future practical use in longitudinal assessments of patients with cancer (Reeve et al. 2013). Future evaluation of minimal clinically important differences is also needed. Although the electronic IQualiV-OG-21 is user-friendly, acceptable, valid and reliable in Brazil, it should be translated and tested in other cultures and languages. To be useful in practice, an instrument needs not only to have good psychometric properties, but it also needs to have equivalent linguistic versions in several populations to be useful in international clinical trials.

Conclusions

We present the first electronic PRO multimedia instrument for oncology developed to measure the HRQOL of Brazilian patients with cancer. The IQualiV-OG-21 can be considered well-valid, and reliable. The electronic form is acceptable and easy to complete, even by patients with low literacy skills.

Declarations

Authors’ contributions

Conceived and designed the experiments: CEP, BSRP, GU. Performed the experiments: CEP, FAFS, GRZ, DAPA, MAB, AAJ, GG, HA, BSRP. Analyzed the data: CEP, EAH, BSRP. Wrote the paper: CEP, FAFS, GRZ, BEM, EAH, GU, BSRP. Approved the final version: CEP, FAFS, GRZ, DAPA, MAB, AAJ, GG, HA, EAH, BSRP. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to thank Dr. José Humberto Fregnani for his critical opinions in the initial design of this study. In addition, the authors thank Ms. Heloisa Pelisser Canton and other assistant and coordinator members of the Research Support Center (Barretos Cancer Hospital) because of their important help in data collection. We are grateful to the researchers Eliane Marçon Barroso and Mayara Goulart de Camargos for their help in the transcription of the patients’ speeches in the qualitative phase of the study. We would like to thank Mr. Alexandre Faustinoni, from ADS Sistemas, because of his participation in the construction of the electronic version of IQualiV-OG-21.

Competing interests

The authors declare that they have competing interests.

Funding

This study was funded by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq, Proc. No 471719/2013-7).

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Divisão de Mama e Ginecologia, Departamento de Oncologia Clínica, Barretos Cancer Hospital, Rua Antenor Duarte Vilella, 1331, Bairro Dr Paulo Prata, Barretos, São Paulo, CEP: 14784-400, Brazil
(2)
Health-Related Quality of Life Research Group (GPQual), Barretos Cancer Hospital, Barretos, São Paulo, 14784-400, Brazil
(3)
Center for Research Support - NAP, Barretos Cancer Hospital, Barretos, São Paulo, 14784-400, Brazil
(4)
Barretos School of Health Sciences, Dr. Paulo Prata - FACISB, Barretos, São Paulo, 14785-002, Brazil
(5)
InORP - Oncology Institute of Ribeirão Preto, Ribeirão Prêto, São Paulo, 14025-270, Brazil
(6)
Hospital das Clínicas de Botucatu, Botucatu, São Paulo, 18618-970, Brazil
(7)
Department of Clinical Oncology, Mater Dei Hospital, Belo Horizonte, Minas Gerais, 30140-093, Brazil
(8)
Department of Medical Social Sciences and Center for Patient-Centered Outcomes, Northwestern University Feinberg School of Medicine, Chicago, IL 60611, USA
(9)
Botucatu Medical School, Universidade Estadual Paulista -UNESP, Botucatu, São Paulo, 18618-970, Brazil

References

  1. Apolinario D, Mansur LL, Carthery-Goulart MT et al (2014) Detecting limited health literacy in Brazil: development of a multidimensional screening tool. Health Promot Int 29:5–14. doi:10.1093/heapro/dat074 View ArticleGoogle Scholar
  2. Bardin L (1994) A análise de conteúdo. Edições 70, LisboaGoogle Scholar
  3. Bond T, Fox C (2007) Applying the Rasch Model: fundamental measurement in the human sciences, 2nd edn. Routledge, New YorkGoogle Scholar
  4. Brabo EP, Paschoal MEM, Biasoli I et al (2006) Brazilian version of the QLQ-LC13 lung cancer module of the European Organization for Research and Treatment of Cancer: preliminary reliability and validity report. Qual Life Res 15:1519–1524. doi:10.1007/s11136-006-0009-9 View ArticleGoogle Scholar
  5. Cella DF, Tulsky DS, Gray G et al (1993) The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J Clin Oncol 11:570–579Google Scholar
  6. Crowne D, Marlowe D (1964) The approval motive: studies in evaluative dependence. Wiley, New YorkGoogle Scholar
  7. Edwards A (1957) Techniques of attitude scale construction. Appleton-Century-Crofts, New YorkView ArticleGoogle Scholar
  8. Fayers P, Machin D (2007) Quality of Life: the assessment, analysis and interpretation of patient-reported outcomes, 2nd edn. Wiley, ChinchesterView ArticleGoogle Scholar
  9. Greenhalgh J (2009) The applications of PROs in clinical practice: what are they, do they work, and why? Qual Life Res 18:115–123. doi:10.1007/s11136-008-9430-6 View ArticleGoogle Scholar
  10. Hahn EA, Cella D (2003) Health outcomes assessment in vulnerable populations: measurement challenges and recommendations. Arch Phys Med Rehabil 84:S35–S42. doi:10.1053/apmr.2003.50245 View ArticleGoogle Scholar
  11. Hahn EA, Cellal D, Dobrez DG et al (2003) Quality of life assessment for low literacy Latinos: a new multimedia program for self-administration. J Oncol Manag 12:9–12Google Scholar
  12. Hahn EA, Cella D, Dobrez D et al (2004) The talking touchscreen: a new approach to outcomes assessment in low literacy. Psychooncology 13:86–95. doi:10.1002/pon.719 View ArticleGoogle Scholar
  13. Hahn EA, Cella D, Dobrez DG et al (2007) The impact of literacy on health-related quality of life measurement and outcomes in cancer outpatients. Qual Life Res 16:495–507. doi:10.1007/s11136-006-9128-6 View ArticleGoogle Scholar
  14. Hearn J, Higginson IJ (1997) Outcome measures in palliative care for advanced cancer patients: a review. J Public Health Med 19:193–199View ArticleGoogle Scholar
  15. Higginson IJ, Carr AJ (2001) Measuring quality of life: using quality of life measures in the clinical setting. BMJ 322:1297–1300View ArticleGoogle Scholar
  16. Hu L, Bentler P (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model A Multidiscip J 6:1–55View ArticleGoogle Scholar
  17. IBGE (2012) Censo Demográfico 2010. ftp://ibge.gov.br/Censos/Censo_Demografico_2010/Caracteristicas_Gerais_Religiao_Deficiencia/caracteristicas_religiao_deficiencia.pdf. Accessed 7 Jun 2013
  18. IBGE (2015) População. http://www.ibge.gov.br/apps/populacao/projecao/index.html
  19. INCA (2014) Estimativa 2014—Incidência de câncer no Brasil. http://www.inca.gov.br/estimativa/2014/
  20. Ishikawa NM, Thuler LCS, Giglio AG et al (2010) Validation of the Portuguese version of functional assessment of cancer therapy-fatigue (FACT-F) in Brazilian cancer patients. Support Care Cancer 18:481–490. doi:10.1007/s00520-009-0697-0 View ArticleGoogle Scholar
  21. Lucchetti G, Lucchetti ALG, de Bernardin Gonçalves JP, Vallada HP (2013) Validation of the Portuguese version of the functional assessment of chronic illness therapy-spiritual well-being scale (FACIT-Sp 12) among brazilian psychiatric inpatients. J Relig Health. doi:10.1007/s10943-013-9785-z Google Scholar
  22. Miles M, Huberman A (1994) Qualitative data analysis: an expanded sourcebook, 2nd edn. Sage, Thousand OaksGoogle Scholar
  23. Paiva CE, Paiva BSR, de Castro RA et al (2011) A pilot study addressing the impact of religious practice on quality of life of breast cancer patients during chemotherapy. J Relig Health. doi:10.1007/s10943-011-9468-6 Google Scholar
  24. Peterman AH, Fitchett G, Brady MJ et al (2002) Measuring spiritual well-being in people with cancer: the functional assessment of chronic illness therapy–Spiritual Well-being Scale (FACIT-Sp). Ann Behav Med 24:49–58View ArticleGoogle Scholar
  25. Reeve BB, Wyrwich KW, Wu AW et al (2013) ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res 22:1889–1905. doi:10.1007/s11136-012-0344-y View ArticleGoogle Scholar
  26. Ribeiro V, Vóvio C, Moura M (2002) Letramento no Brasil: alguns resultados do indicador nacional de alfabetismo funcional. Educ Soc Campinas 23:49–70View ArticleGoogle Scholar
  27. Schermelleh-Engel K, Moosbrugger H, Muller H (2003) Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res 8:23–74Google Scholar
  28. Selltiz C, Wrightsman L, Cook S (1976) Research methods in social relations. Holt, Rinehart and Winston, New YorkGoogle Scholar
  29. Terwee CB, Bot SDM, de Boer MR et al (2007) Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 60:34–42. doi:10.1016/j.jclinepi.2006.03.012 View ArticleGoogle Scholar
  30. Thumboo J, Wee H-L, Cheung Y-B et al (2006) Development of a Smiling Touchscreen multimedia program for HRQoL assessment in subjects with varying levels of literacy. Value Health 9:312–319. doi:10.1111/j.1524-4733.2006.00120.x View ArticleGoogle Scholar
  31. Van der Linden WJ, Hambleton R (1997) Handbook of modern item response theory. Springer, New YorkView ArticleGoogle Scholar
  32. Yost KJ, Webster K, Baker DW et al (2010) Acceptability of the talking touchscreen for health literacy assessment. J Health Commun 15(Suppl 2):80–92. doi:10.1080/10810730.2010.500713 View ArticleGoogle Scholar

Copyright

© The Author(s) 2016

Advertisement