Mixed methods research in complementary and alternative medicine: a scoping review

doi:10.19852/j.cnki.jtcm.20220602.002

Abstract

Abstract:

OBJECTIVE: To update the current characteristics about the scope and quality of mixed methods research (MMR) in complementary and alternative medicine (CAM) after nearly 10 years. METHODS: A 5-stage approach for conducting a scoping review was adopted. Articles published on the top 10 journals in CAM with the highest impact factor in 2020 were screened for MMR. Information of included articles were extracted, and then synthesized to illustrate the current state. Methodological quality was evaluated according to the Mixed Method Appraisal Tool (MMAT) 2018 version. RESULTS: A total of 55 (55/2991, 2%) articles using mixed methods were retrieved, including 17 medical studies and 38 ethnobotanical studies. We performed an in-depth analysis on the 17 medical studies, which studied cancer, stress, pain, fatigue, exercises, mindfulness intervention, herbal medicine use, art and acupuncture. Thirteen pilot studies applied MMR to evaluate the feasibility of interventions or programs (13/17, 76%); phenomenology was inferred as the most common philosophical assumptions (13/17, 76%); the most applied type of MMR was convergent design (16/17, 94%); integration often took place at integration (12/17, 71%). Among the 16 eligible studies for quality appraisal, majority were rated as good (14/16, 88%), whereas two studies were rated as poorly described. Primarily, a poor rating was due to incomplete reporting of data analysis and citations in qualitative components; lack of confounder controlling and the sampling strategy in quantitative components; poor description of integration and justification for mixed methods. Comparing with the previous review, fewer MMR were published in 2020 in CAM, but the proportion of studies that clearly reported MMR has increased by 4 times (4%→15%). CONCLUSION: CAM researchers need to realize the benefits that MMR can have on conducting further health care research. Our findings highlight that applying MMR will be helpful to understand the complex dynamics and interdisciplinary nature of complex intervention. In addition, addressing a standardized reporting criteria for MMR is recommended.

Key words: complementary therapies, review, biomedical research, pilot projects, mixed methods

DU Wanqing, WANG Liuding, JIA Min, LIANG Xiao, LI Bo, ZHANG Yunling, LIAO Xing. Mixed methods research in complementary and alternative medicine: a scoping review[J]. Journal of Traditional Chinese Medicine, 2022, 42(4): 652-666.

Figures/Tables 4

References 71

1	Palinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health 2019; 40: 423-42. DOI URL
2	Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 3rd ed. CA: Sage Publications, 2017: 34.
3	Wisdom JP, Cavaleri MA, Onwuegbuzie AJ, Green CA. Methodological reporting in qualitative, quantitative, and mixed methods health services research articles. Health Services Res 2012; 47: 721-45. DOI URL
4	Fielding NG. Triangulation and mixed methods designs: data integration with new research technologies. J Mixed Methods Res 2012; 6: 124-36. DOI URL
5	Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health 2014; 35: 29-45. DOI URL
6	Curry LA, Krumholz HM, O'Cathain A, Plano Clark VL, Cherlin E, Bradley EH. Mixed methods in biomedical and health services research. Circ Cardiovasc Qual Outcomes 2013; 6: 119-23. DOI URL
7	Fàbregues S, Hong QN, Escalante-Barrios EL, Guetterman TC, Meneses J, Fetters MD. A methodological review of mixed methods research in palliative and end-of-life care (2014-2019). Int J Environ Res Public Health 2020; 17: 3853. DOI URL
8	Sharma S, Rafferty AM, Boiko O. The role and contribution of nurses to patient flow management in acute hospitals: a systematic review of mixed methods studies. Int J Nurs Stud 2020; 110: 103709. DOI URL
9	De Allegri M, Sieleunou I, Abiiro GA, Ridde V. How far is mixed methods research in the field of health policy and systems in Africa? A scoping review. Health Policy Plan 2018; 33: 445-55. DOI URL
10	Maudsley G. Mixing it but not mixed-up: mixed methods research in medical education (a critical narrative review). Med Teach 2011; 33: e92-104.
11	Bishop FL, Holmes MM. Mixed methods in CAM research:a systematic review of studies published in 2012. Evid Based Complement Alternat Med 2013; 2013: 187365.
12	Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. Bmj 2009; 339: b3496. DOI URL
13	World Health Organization. WHO global report on traditional and complementary medicine 2019. Geneva: World Health Organization, 2019: 10-20.
14	Ahn AC, Nahin RL, Calabrese C, et al. Applying principles from complex systems to studying the efficacy of CAM therapies. J Altern Complement Med 2010; 16: 1015-22. DOI URL
15	Veziari Y, Leach MJ, Kumar S. Barriers to the conduct and application of research in complementary and alternative medicine: a systematic review. BMC Complement Altern Med 2017; 17: 166. DOI URL
16	Low MY, Lacson C, Zhang F, Kesslick A, Bradt J. Vocal music therapy for chronic pain: a mixed methods feasibility study. J Altern Complement Med 2020; 26: 113-22. DOI URL
17	Osypiuk K, Ligibel J, Giobbie-Hurder A, et al. Qigong mind-body exercise as a biopsychosocial therapy for persistent post-surgical pain in breast cancer: a pilot study. Integr Cancer Ther 2020; 19: 1534735419893766.
18	Nsibirwa S, Anguzu G, Kamukama S, Ocama P, Nankya-Mutyoba J. Herbal medicine use among patients with viral and non-viral Hepatitis in Uganda: prevalence, patterns and related factors. BMC Complement Med Ther 2020; 20: 169. DOI PMID
19	Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Social Res Methodology 2005; 8: 19-32. DOI URL
20	Rosenfield PL. The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences. Soc Sci Med 1992; 35: 1343-57. PMID
21	Sandelowski M. Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Res Nurs Health. 2000; 23: 246-55. PMID
22	Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res 2013; 48: 2134-56. DOI URL
23	Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M. Mixed Methods Appraisal Tool (MMAT), version 2018. Canadian Intellectual Property Office #1148552, Industry Canada, 2018 Aug 1st.
24	Reporting the results of the MMAT. Mixed methods appraisal tool public, 2020-12-09, cited 2022-05-27;1: Available from URL: http://mixedmethodsappraisaltoolpublic.pbworks.com/w/browse/#view=ViewAllObjects.
25	Attia M, Edge J. Be(com)ing a reflexive researcher: a developmental approach to research methodology. Open Rev Edu Res 2017; 4: 33-45.
26	Abdusalam A, Zhang Y, Abudoushalamu M, et al. Documenting the heritage along the Silk Road: An ethnobotanical study of medicinal teas used in Southern Xinjiang, China. J Ethnopharmacol 2020; 260: 113012. DOI URL
27	Ahoyo CC, Houéhanou TD, Yaoïtcha AS, et al. Traditional medicinal knowledge of woody species across climatic zones in Benin (West Africa). J Ethnopharmacol 2021; 265:113417. DOI URL
28	Alqethami A, Aldhebiani AY, Teixidor-Toneu I. Medicinal plants used in Jeddah, Saudi Arabia: a gender perspective. J Ethnopharmacol 2020; 257: 112899.
29	Antonella Carabajal MP, Perea MC, Isla MI, Zampini IC. The use of jarilla native plants in a Diaguita-Calchaquí indigenous community from northwestern Argentina: an ethnobotanical, phytochemical and biological approach. J Ethnopharmacol 2020; 247: 112258.
30	Anywar G, Kakudidi E, Byamukama R, Mukonzo J, Schubert A, Oryem-Origa H. Indigenous traditional knowledge of medicinal plants used by herbalists in treating opportunistic infections among people living with HIV/AIDS in Uganda. J Ethnopharmacol 2020; 246: 112205.
31	Ataba E, Katawa G, Ritter M, et al. Ethnobotanical survey, anthelmintic effects and cytotoxicity of plants used for treatment of helminthiasis in the Central and Kara regions of Togo. BMC Complement Med Ther 2020; 20: 212. DOI URL
32	Baldé MA, Tuenter E, Traoré MS, et al. Antimicrobial investigation of ethnobotanically selected guinean plant species. J Ethnopharmacol 2020; 263: 113232. DOI URL
33	Barbosa F, Hlashwayo D, Sevastyanov V, et al. Medicinal plants sold for treatment of bacterial and parasitic diseases in humans in Maputo city markets, Mozambique. BMC Complement Med Ther 2020; 20: 19. DOI URL
34	Chamorro MF, Ladio A. Native and exotic plants with edible fleshy fruits utilized in Patagonia and their role as sources of local functional foods. BMC Complement Med Ther 2020; 20: 155. DOI URL
35	Dassou GH, Ouachinou JMS, Adomou AC, et al. Plant and natural product based homemade remedies for veterinary uses by the Peul community in Benin. J Ethnopharmacol 2020; 261: 113107. DOI URL
36	Dhakal P, Chettri B, Lepcha S, Acharya BK. Rich yet undocumented ethnozoological practices of socio-culturally diverse indigenous communities of Sikkim Himalaya, India. J Ethnopharmacol 2020; 249: 112386. DOI URL
37	Arden-Close E, Mitchell F, Davies G, et al. Mindfulness-based interventions in recurrent ovarian cancer: a mixed-methods feasibility study. Integr Cancer Ther 2020; 19: 1534735420908341.
38	Bloom-Foster J, Mehl-Madrona L. An ultra-brief mindfulness-based intervention for patients in treatment for opioid addiction with buprenorphine: a primary care feasibility pilot study. J Altern Complement Med 2020; 26: 34-43. DOI URL
39	Conte E, Legacy M, Psihogios A, et al. A prospective outcomes pilot evaluation of inspire now: a program for people with lung cancer. Integr Cancer Ther 2020; 19: 1534735420983472.
40	Hirschberg R, Sylvia LG, Wright EC, et al. Collaborative songwriting intervention for veterans with post-traumatic stress disorder. J Altern Complement Med 2020; 26: 198-203. DOI URL
41	Kaimal G, Carroll-Haskins K, Mensinger JL, Dieterich-Hartwell R, Biondo J, Levin WP. Outcomes of therapeutic artmaking in patients undergoing radiation oncology treatment: a mixed-methods pilot study. Integr Cancer Ther 2020; 19: 1534735420912835.
42	McDonnell KK, Gallerani DG, Newsome BR, et al. A prospective pilot study evaluating feasibility and preliminary effects of breathe easier: a mindfulness-based intervention for survivors of lung cancer and their family members (Dyads). Integr Cancer Ther 2020; 19: 1534735420969829.
43	Mehl A, Reif M, Zerm R, et al. Impact of a multimodal and combination therapy on self-regulation and internal coherence in german breast cancer survivors with chronic cancer-related fatigue: a mixed-method comprehensive cohort design study. Integr Cancer Ther 2020; 19: 1534735420935618.
44	Peters BC, Fields BE, Erdman EA. Hippotherapy practice and safety patterns in the united states: a descriptive survey study. J Altern Complement Med 2020; 26: 743-49. DOI URL
45	Rahayu YYS, Araki T, Rosleine D. Factors affecting the use of herbal medicines in the universal health coverage system in Indonesia. J Ethnopharmacol 2020; 260: 112974. DOI URL
46	Rothman JM, Bilici N, Mergler B, et al. A culinary medicine elective for clinically experienced medical students: a pilot study. J Altern Complement Med 2020; 26: 636-44. DOI URL
47	Shewamene Z, Dune T, Smith CA. Use of traditional and complementary medicine for maternal health and wellbeing by African migrant women in Australia: a mixed method study. BMC Complement Med Ther 2020; 20: 60. DOI PMID
48	Stritter W, Rutert B, Eggert A, Längler A, Holmberg C, Seifert G. Evaluation of an integrative care program in pediatric oncology. Integr Cancer Ther 2020; 19: 1534735420928393.
49	Sylvia L, West E, Blackburn AM, et al. Acceptability of an adjunct equine-assisted activities and therapies program for veterans with posttraumatic stress disorder and/or traumatic brain injury. J Integr Med 2020; 18: 169-73. DOI URL
50	Zhao Q, Zheng S, Delaney GP, et al. Acupuncture for cancer related pain: protocol for a pragmatic randomised wait-list controlled trial. Integr Cancer Ther 2020; 19: 1534735420976579.
51	Zörgő S, Peters GY, Mkhitaryan S. Attitudes Underlying Reliance on Complementary and Alternative Medicine. Integr Cancer Ther 2020; 19: 1534735420910472.
52	Curry LA, Nembhard IM, Bradley EH. Qualitative and mixed methods provide unique contributions to outcomes research. Circulation 2009; 119: 1442-52. DOI URL
53	Creswell JW, Klassen WA, Clark V, Smith KC. Best practices for mixed methods research in the health sciences. Qualitative Social Work 2013; 12: 541-45. DOI URL
54	Alex, Broom. Using qualitative interviews in CAM research: A guide to study design, data collection and data analysis. Complement Ther Med 2005; 13: 65-73. PMID
55	Campbell DJ, Tam-Tham H, Dhaliwal KK, et al. Use of mixed methods research in research on coronary artery disease, diabetes mellitus, and hypertension: a scoping review. Circ Cardiovasc Qual Outcomes 2017; 10: e003310.
56	Tariq S, Woodman J. Using mixed methods in health research. Jrsm Short Reports 2013; 4: 2042533313479197.
57	Curry L, Nunez-Smith M. Mixed methods in health sciences research, CA:Sage Publications, 2014: 37-44.58. Schultes RE, CReis SV. Ethnobotany:evolution of a discipline. Portland: Dioscorides Press, 1995: 264-83.
59	Thangaraj P. Pharmacological assays of plant-based natural products. Switzerland: Springer, 2016: 10-12.
60	O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy 2008; 13: 92-8. DOI PMID
61	Irvine FE, Clark MT, Efstathiou N, et al. The state of mixed methods research in nursing: A focused mapping review and synthesis. J Adv Nurs 2020; 76: 2798-809. DOI URL
62	Farquhar MC, Ewing G, Booth S. Using mixed methods to develop and evaluate complex interventions in palliative care research. Palliat Med 2011; 25: 748-57. DOI PMID
63	Arnold DM, Burns KE, Adhikari NK, Kho ME, Meade MO, Cook DJ. The design and interpretation of pilot trials in clinical research in critical care. Crit Care Med 2009; 37(1 Suppl): S69-74. DOI URL
64	In J. Introduction of a pilot study. Korean J Anesthesiol 2017; 70: 601-5. DOI URL
65	Gaglio B, Henton M, Barbeau A, et al. Methodological standards for qualitative and mixed methods patient centered outcomes research. Bmj 2020; 371: m4435.
66	Adam S, Thong MSY, Martin-Diener E, et al. Identifying classes of the pain, fatigue, and depression symptom cluster in long-term prostate cancer survivors-results from the multi-regional Prostate Cancer Survivorship Study in Switzerland (PROCAS). Support Care Cancer 2021; 1: 11.
67	Creswell JW, Poth CN. Qualitative inquiry & research design-choosing among five approaches. 4th ed. London: SAGE Publications, 2018: 30-5.
68	Yan H, Karmur BS, Kulkarni AV. Comparing effects of treatment: controlling for confounding. Neurosurgery 2020; 86: 325-31. DOI URL
69	Tashakkori A, Teddlie C. Mixed methodology: combining qualita-tive and quantitative approaches: SAGE Publications 1998: 30-5.
70	Plano Clark VL. The adoption and practice of mixed methods: u.s. trends in federally funded health-related research. Qualitative Inquiry 2010; 16: 428-40. DOI URL
71	Curry LA, Nembhard IM, Bradley EH. Qualitative and mixed methods provide unique contributions to outcomes research. Circulation 2009; 119: 1442-5 DOI URL

Study		Journal	Disease/ Condition		Field/ Discipline			Country	Sample size		Study focus	Justification for MMR		Qualitative metho-dology	Qualitative study design	Quantitative study design		Mixed methods design		Priority		Time of inte-gration		Overall quality
Low		JACM	Chronic pain		Music therapy			USA	43 patients		Evaluation of the feasibility and effects of intervention	Comple-mentarity^a		Pheno-menology	Semi-structured interview	Prospective, randomized controlled trial		Conver-gent		QUAN		Interpr-etation		****
Arden-close		ICT	Stress in cancer		Mindful-ness intervention			UK	28 patients		Evaluation of the feasibility of intervention	Comple-mentarity^a		Pheno-menology	Focus group	Prospective, pre-post study		Conver-gent^a		QUAN		Interpr-etation		****
Kaimal		ICT	Stress in cancer		Artmaking			USA	22 patients		Evaluation of interventions	Comple-mentarity^a		Ethno-graphy	Open-ended survey, field notes, and artwork	Prospective, non-ran-domized trial		Conver-gent		QUAN		Interpr-etation		*****
Mehl		ICT	Fatigue in cancer		Multimodal and combina-tion therapy			Germany	105 patients		Evaluation of the effect of interventions	Comple-mentarity^a		Pheno-menology	Opended question-naire	Prospective, pragmatic randomized controlled trial, 3-arm, 3-center, open-label		Conver-gent^a		QUAN		Inter-pretation		***
Osypiuk		ICT	Post-surgical pain in cancer		Qigong exercise			USA	21 patients		Evaluation of the feasibility, safety, and effectiveness of intervention	Comple-mentarity^a		Pheno-menology	Semi-structured interview	Prospective, pre-post study		Conver-gent^a		QUAN		Analysis		****
Stritter		ICT	Pediatric oncology		Care program			Germany	69 patients +5 nurses		Evaluation of the imp-lementation and factors of inter-vention	Trian-gulation^a		Ethno-graphy	Field notes of partici-pant obser-vations and informal conver-sations	Survey		Conver-gent^a		QUAL		Inter-pretation		***
Shewamene		BMC CMT	Maternity		Use of com-plementary medicine			Australia	319 pa-tients (15 interviewed)		Description of the experiences, perspectives and factors of CAM use	Deve-lopment^a		Ethno-graphy	In-depth interview	Survey		Sequential explanatory		QUAN		Study design		****
Bloom-foster		JACM	Opioid addiction		Mindful-ness intervention			USA	40 patients		Evaluation of the feasibility and accep-tability of intervention	Comple-mentarity^a		Pheno-menology	Semi-structured interview	Prospective, single-arm cohort study		Conver-gent^a		QUAN		Inter-pretation		****
Study	Journal		Disease/ Con- dition		Field/ Dis-cipline	Country		Sample size		Study focus	Justification for MMR	Qualitative metho-dology		Qualitative study design		Quantitative study design		Mixed methods design		Priority		Time of integration		Overall quality
Conte	ICT		Cancer		A prog-ram involved lecture and yoga	Canada		9 patients		Evaluation of the feasibility and effect-tiveness of intervention	Complementa-rity^a	Pheno-menology		Semi-structured interview and open-ended questionnaire		Prospective, prepost study		Conver-gent^a		QUAN		Inter-pretation		***
Hirschberg	JACM		Post-traumatic stress disorder		Song-writing	USA		10 patients		Evaluation of the fea-sibility, accep-tability and effectiveness of intervention	Complementarity^a	Pheno-menology		Semi-structured interview		Prospective, prepost study		Conver-gent^a		QUAN		Inter-pretation		***
Mcdonnell	ICT		Cancer		Mindfulness inter-vention	USA		62 patients+ relatives		Evaluation of the fea-sibility and effect of intervention	Complementarity^a	Pheno-menology		Interview		Prospective, prepost study		Conver-gent^a		QUAN		Inter-pretation		*****
Peters	JACM		/		Hippotherapy use	USA		92 clinicians		Description of the safety and practice patt-erns of the use of a therapy	Complementary^a	Pheno-menology		Open-ended questionnaire		Survey		Conver-gent^a		QUAN		Analysis		**
Rothman	JACM		/		Culinary course for medical student	USA		31 medical students		Evaluation on the feasibility and effect of intervention	Complementarity^a	Pheno-menology		Open-ended questionnaire		Survey		Conver-gent^a		QUAN		Inter-pretation		**
Sylvia	JIM		Post-traumatic stress or brain injury		Equine therapy-	USA		62 veterans+44 relatives		Evaluation on the accep-tability of intervention	Complementary^a	Phenomenology		Open-ended survey		Survey		Conver-gent^a		QUAN		Inter-pretation		***
Nsibirwa	BMC CMT		Hepatitis		Herbal medicine use	Uganda		310 patients		Description of the preva-lence, patt-erns and factors of herbal medicine use	Complementary^a	Pheno-menology		Focus group		Cross-sectional study		Conver-gent^a		QUAN		Analysis		****
Rahayu	JEP		/		Herbal medicine use		Indonesia		634 house-holds	Description of the patterns and factors of herbal medicine use	Complementary^a		Pheno-menology	Semi-structured interview		Survey		Conver-gent^a		QUAN		Analysis		***
Study	Journal			Disease/ Condition	Field/ Discipline		Country		Sample size	Study focus	Justification for MMR		Qualitative metho-dology		Qualitative study design		Quantitative study design		Mixed methods design		Priority		Time of inte-gration		Overall quality
Zhao	ICT			Pain in cancer	Acupuncture		Australia		106 patients	(Protocol) Evaluation on the effectiveness, safety and experience of intervention	Complementarity^a		Pheno-menology		Interview		Prospec-tive, prag-matic ran-domized controlled trial		Conver-gent^a		QUAN		Inter-pretation		Protocol

Study		Journal	Disease/ Condition		Field/ Discipline			Country	Sample size		Study focus	Justification for MMR		Qualitative metho-dology	Qualitative study design	Quantitative study design		Mixed methods design		Priority		Time of inte-gration		Overall quality
Low		JACM	Chronic pain		Music therapy			USA	43 patients		Evaluation of the feasibility and effects of intervention	Comple-mentarity^a		Pheno-menology	Semi-structured interview	Prospective, randomized controlled trial		Conver-gent		QUAN		Interpr-etation		****
Arden-close		ICT	Stress in cancer		Mindful-ness intervention			UK	28 patients		Evaluation of the feasibility of intervention	Comple-mentarity^a		Pheno-menology	Focus group	Prospective, pre-post study		Conver-gent^a		QUAN		Interpr-etation		****
Kaimal		ICT	Stress in cancer		Artmaking			USA	22 patients		Evaluation of interventions	Comple-mentarity^a		Ethno-graphy	Open-ended survey, field notes, and artwork	Prospective, non-ran-domized trial		Conver-gent		QUAN		Interpr-etation		*****
Mehl		ICT	Fatigue in cancer		Multimodal and combina-tion therapy			Germany	105 patients		Evaluation of the effect of interventions	Comple-mentarity^a		Pheno-menology	Opended question-naire	Prospective, pragmatic randomized controlled trial, 3-arm, 3-center, open-label		Conver-gent^a		QUAN		Inter-pretation		***
Osypiuk		ICT	Post-surgical pain in cancer		Qigong exercise			USA	21 patients		Evaluation of the feasibility, safety, and effectiveness of intervention	Comple-mentarity^a		Pheno-menology	Semi-structured interview	Prospective, pre-post study		Conver-gent^a		QUAN		Analysis		****
Stritter		ICT	Pediatric oncology		Care program			Germany	69 patients +5 nurses		Evaluation of the imp-lementation and factors of inter-vention	Trian-gulation^a		Ethno-graphy	Field notes of partici-pant obser-vations and informal conver-sations	Survey		Conver-gent^a		QUAL		Inter-pretation		***
Shewamene		BMC CMT	Maternity		Use of com-plementary medicine			Australia	319 pa-tients (15 interviewed)		Description of the experiences, perspectives and factors of CAM use	Deve-lopment^a		Ethno-graphy	In-depth interview	Survey		Sequential explanatory		QUAN		Study design		****
Bloom-foster		JACM	Opioid addiction		Mindful-ness intervention			USA	40 patients		Evaluation of the feasibility and accep-tability of intervention	Comple-mentarity^a		Pheno-menology	Semi-structured interview	Prospective, single-arm cohort study		Conver-gent^a		QUAN		Inter-pretation		****
Study	Journal		Disease/ Con- dition		Field/ Dis-cipline	Country		Sample size		Study focus	Justification for MMR	Qualitative metho-dology		Qualitative study design		Quantitative study design		Mixed methods design		Priority		Time of integration		Overall quality
Conte	ICT		Cancer		A prog-ram involved lecture and yoga	Canada		9 patients		Evaluation of the feasibility and effect-tiveness of intervention	Complementa-rity^a	Pheno-menology		Semi-structured interview and open-ended questionnaire		Prospective, prepost study		Conver-gent^a		QUAN		Inter-pretation		***
Hirschberg	JACM		Post-traumatic stress disorder		Song-writing	USA		10 patients		Evaluation of the fea-sibility, accep-tability and effectiveness of intervention	Complementarity^a	Pheno-menology		Semi-structured interview		Prospective, prepost study		Conver-gent^a		QUAN		Inter-pretation		***
Mcdonnell	ICT		Cancer		Mindfulness inter-vention	USA		62 patients+ relatives		Evaluation of the fea-sibility and effect of intervention	Complementarity^a	Pheno-menology		Interview		Prospective, prepost study		Conver-gent^a		QUAN		Inter-pretation		*****
Peters	JACM		/		Hippotherapy use	USA		92 clinicians		Description of the safety and practice patt-erns of the use of a therapy	Complementary^a	Pheno-menology		Open-ended questionnaire		Survey		Conver-gent^a		QUAN		Analysis		**
Rothman	JACM		/		Culinary course for medical student	USA		31 medical students		Evaluation on the feasibility and effect of intervention	Complementarity^a	Pheno-menology		Open-ended questionnaire		Survey		Conver-gent^a		QUAN		Inter-pretation		**
Sylvia	JIM		Post-traumatic stress or brain injury		Equine therapy-	USA		62 veterans+44 relatives		Evaluation on the accep-tability of intervention	Complementary^a	Phenomenology		Open-ended survey		Survey		Conver-gent^a		QUAN		Inter-pretation		***
Nsibirwa	BMC CMT		Hepatitis		Herbal medicine use	Uganda		310 patients		Description of the preva-lence, patt-erns and factors of herbal medicine use	Complementary^a	Pheno-menology		Focus group		Cross-sectional study		Conver-gent^a		QUAN		Analysis		****
Rahayu	JEP		/		Herbal medicine use		Indonesia		634 house-holds	Description of the patterns and factors of herbal medicine use	Complementary^a		Pheno-menology	Semi-structured interview		Survey		Conver-gent^a		QUAN		Analysis		***
Study	Journal			Disease/ Condition	Field/ Discipline		Country		Sample size	Study focus	Justification for MMR		Qualitative metho-dology		Qualitative study design		Quantitative study design		Mixed methods design		Priority		Time of inte-gration		Overall quality
Zhao	ICT			Pain in cancer	Acupuncture		Australia		106 patients	(Protocol) Evaluation on the effectiveness, safety and experience of intervention	Complementarity^a		Pheno-menology		Interview		Prospec-tive, prag-matic ran-domized controlled trial		Conver-gent^a		QUAN		Inter-pretation		Protocol

Categories of study design				Study	Low^a		Mehl^a		Arden-close^a		Kaimal^a			Osy-piuk^a	Bloom-foster		Conte		Hirs-chberg	Mcd-onnell	Stritter^a	Peters	Roth man		Sylvia			Shewa-mene^a		Nsibirwa			Rahayu
QUAL				1.1. Is the qualitative approach appropriate to answer the research question?	1		0		1		1			1	1		1		1	1	1	1	1		1			1		1			1
				1.2. Are the qualitative data collection methods adequate to address the research question?	1		1		1		1			1	1		1		1	1	1	0	0		1			1		1			1
				1.3. Are the findings adequately derived from the data?	1		0		1		1			1	1		0		1	1	1	0	0		1			1		1			0
				1.4. Is the interpretation of results sufficiently substantiated by data?	1		1		1		1			1	1		0		1	1	1	0	0		1			1		1			0
				1.5. Is there coherence between qualitative data sources, collection, analysis and interpretation?	1		1		1		1			1	1		1		1	1	1	1	1		1			1		1			1
QUAN RCT				2.1. Is randomization appropriately performed?	1		1
				2.2. Are the groups comparable at baseline?	1		1
				2.3. Are there complete outcome data?	1		1
				2.4. Are outcome assessors blinded to the intervention provided?	0		0
				2.5 Did the participants adhere to the assigned intervention?	1		0
QUAN Non-rand-omized				3.1. Are the participants representative of the target population?					1		1			1	0		0		0	1
				3.2. Are measurements appropriate regarding both the outcome and intervention (or exposure)?					1		1		1		1		1		1	1
Categories of study design			Study	Low^a	Mehl^a		Arden-close^a	Kaimal^a			Osy-piuk^a	Bloom-foster			Conte	Hirs-chberg		Mcdon-nell	Stritter^a	Peters		Rothman		Sylvia			Shewamene^a		Nsibirwa			Rahayu
QUAN Non-ran-domized			3.3. Are there complete outcome data?				1	1			1	1			1	1	1
			3.4. Are the confounders accounted for in the design and analysis?				0	1			1	1			0	0	1
			3.5During the study period, is the intervention administered (or exposure occurred) as intended?				1	1			1	1			1	1	1
QUAN descriptive		4.1. Is the sampling strategy relevant to address the research question?																	NM	1		NM		NM		1			1			NM
		4.2. Is the sample representative of the target population?																	1	NM		NM		0		1			1			NM
		4.3. Are the measurements appropriate?																	1	1		0		1		1			1			1
		4.4. Is the risk of nonresponse bias low?																	NM	NM		1		1		0			NM			NM
		4.5. Is the statistical analysis appropriate to answer the research question?																	1	1		1		1		1			1			1
Mixed Meth-ods		5.1. Is there an adequate rationale for using a Mixed Methods design to address the research question?		1	0		1	1			1	1			1	0	1		0	0		0		0	1						1			0
Categories of study design		Study			Low^a	Mehl^a		Arden-close^a		Kaimal^a			Osypiuk^a		Bloom-foster		Conte		Hirsch-berg	Mcdon-nell	Stritter^a	Peters		Rothman	Sylvia				Shewamene^a		Nsibirwa			Rahayu
	5.2. Are the different components of the study effectively integrated to answer the research question?				1	1		1		1			1		1		0		1	1	1	1		1	1				1		1			1
	5.3. Are the outputs of the integration of qualitative and quantitative components adequately interpreted?				1	1		1		1			1		1		1		1	1	1	1		1	1				1		1			1
	5.4. Are divergences and inconsistencies between quantitative and qualitative results adequately addressed?				1	1		1		1			1		1		1		1	1	1	1		1	1				1		1			1
	5.5. Do the different components of the study adhere to the quality criteria of each tradition of the methods involved?				1	0		0		1			1		1		0		0	1	0	0		0	0				1		1			0
Overall quality (lowest score of the study components)					****	***		****		*****			*****		****		***		***	*****	***	**		**	***				****		****			***

Categories of study design				Study	Low^a		Mehl^a		Arden-close^a		Kaimal^a			Osy-piuk^a	Bloom-foster		Conte		Hirs-chberg	Mcd-onnell	Stritter^a	Peters	Roth man		Sylvia			Shewa-mene^a		Nsibirwa			Rahayu
QUAL				1.1. Is the qualitative approach appropriate to answer the research question?	1		0		1		1			1	1		1		1	1	1	1	1		1			1		1			1
				1.2. Are the qualitative data collection methods adequate to address the research question?	1		1		1		1			1	1		1		1	1	1	0	0		1			1		1			1
				1.3. Are the findings adequately derived from the data?	1		0		1		1			1	1		0		1	1	1	0	0		1			1		1			0
				1.4. Is the interpretation of results sufficiently substantiated by data?	1		1		1		1			1	1		0		1	1	1	0	0		1			1		1			0
				1.5. Is there coherence between qualitative data sources, collection, analysis and interpretation?	1		1		1		1			1	1		1		1	1	1	1	1		1			1		1			1
QUAN RCT				2.1. Is randomization appropriately performed?	1		1
				2.2. Are the groups comparable at baseline?	1		1
				2.3. Are there complete outcome data?	1		1
				2.4. Are outcome assessors blinded to the intervention provided?	0		0
				2.5 Did the participants adhere to the assigned intervention?	1		0
QUAN Non-rand-omized				3.1. Are the participants representative of the target population?					1		1			1	0		0		0	1
				3.2. Are measurements appropriate regarding both the outcome and intervention (or exposure)?					1		1		1		1		1		1	1
Categories of study design			Study	Low^a	Mehl^a		Arden-close^a	Kaimal^a			Osy-piuk^a	Bloom-foster			Conte	Hirs-chberg		Mcdon-nell	Stritter^a	Peters		Rothman		Sylvia			Shewamene^a		Nsibirwa			Rahayu
QUAN Non-ran-domized			3.3. Are there complete outcome data?				1	1			1	1			1	1	1
			3.4. Are the confounders accounted for in the design and analysis?				0	1			1	1			0	0	1
			3.5During the study period, is the intervention administered (or exposure occurred) as intended?				1	1			1	1			1	1	1
QUAN descriptive		4.1. Is the sampling strategy relevant to address the research question?																	NM	1		NM		NM		1			1			NM
		4.2. Is the sample representative of the target population?																	1	NM		NM		0		1			1			NM
		4.3. Are the measurements appropriate?																	1	1		0		1		1			1			1
		4.4. Is the risk of nonresponse bias low?																	NM	NM		1		1		0			NM			NM
		4.5. Is the statistical analysis appropriate to answer the research question?																	1	1		1		1		1			1			1
Mixed Meth-ods		5.1. Is there an adequate rationale for using a Mixed Methods design to address the research question?		1	0		1	1			1	1			1	0	1		0	0		0		0	1						1			0
Categories of study design		Study			Low^a	Mehl^a		Arden-close^a		Kaimal^a			Osypiuk^a		Bloom-foster		Conte		Hirsch-berg	Mcdon-nell	Stritter^a	Peters		Rothman	Sylvia				Shewamene^a		Nsibirwa			Rahayu
	5.2. Are the different components of the study effectively integrated to answer the research question?				1	1		1		1			1		1		0		1	1	1	1		1	1				1		1			1
	5.3. Are the outputs of the integration of qualitative and quantitative components adequately interpreted?				1	1		1		1			1		1		1		1	1	1	1		1	1				1		1			1
	5.4. Are divergences and inconsistencies between quantitative and qualitative results adequately addressed?				1	1		1		1			1		1		1		1	1	1	1		1	1				1		1			1
	5.5. Do the different components of the study adhere to the quality criteria of each tradition of the methods involved?				1	0		0		1			1		1		0		0	1	0	0		0	0				1		1			0
Overall quality (lowest score of the study components)					****	***		****		*****			*****		****		***		***	*****	***	**		**	***				****		****			***

Item	2012	2020
Articles published in top 10 CAM journals	2349	2991
Mixed Methods studies [n (%)]	80 (3)	55 (2)
Medical research in MMR [n (%)]	28/80 (35)	17/55 (31)
Ethnobotanical study in MMR [n (%)]	52/80 (65)	38/55 (69)
Studies explicitly mentioned MMR [n (%)]	3 (4)	8 (15)
Quality Appraisal Tool	MMAT 2011 version: 4×4+3 items	MMAT 2018 version: 5×5 items
Main Limitations in MMR studies	Lack of philosophical tension, analytic methods, reflexivity, sampling procedures, and recruitment procedures.	Lack of reporting analyzing process and citations, confounder controlling, sampling strategy, integration and justification for mixed method

Home

Online First

Author Center

Reviewer Center

Issues

Announcement

About Us