Table 3 Quality appraisal of articles included in review with JBI and MMAT tools

Criteria for assessment in JBI

JBI for quasi-experimental

Study

Is it clear in the study what is the ‘cause’ and what is the ‘effect’?

Were the participants included in any comparisons similar?

Were the participants included in any comparisons receiving similar treatment/care, other than the exposure or intervention of interest?

Was there a control group?

Were there multiple measurements of the outcome both pre and post the intervention/exposure?

Was follow up complete and if not, were differences between groups in terms of their follow up adequately described and analyzed?

Were the outcomes of participants included in any comparisons measured in the same way?

Were outcomes measured in a reliable way?

Was appropriate statistical analysis used?

JBI for case control

Study

Were the groups comparable other than the presence of disease in cases or the absence of disease in controls?

Were the groups comparable other than the presence of disease in cases or the absence of disease in controls?

Were the same criteria used for identification of cases and controls?

Was exposure measured in a standard, valid and reliable way?

Was exposure measured in the same way for cases and controls?

Were confounding factors identified?

Were strategies to deal with confounding factors stated?

Were outcomes assessed in a standard, valid and reliable way for cases and controls?

Was the exposure period of interest long enough to be meaningful?

Was appropriate statistical analysis used?

JBI for analytical cohort

Study

Were the two groups similar and recruited from the same population?

Were the exposures measured similarly to assign people to both exposed and unexposed groups?

Was the exposure measured in a valid and reliable way?

Were confounding factors identified?

Were strategies to deal with confounding factors stated?

Were the groups/participants free of the outcome at the start of the study?

Were the outcomes measured in a valid and reliable way?

Was the follow up time reported and sufficient to be long enough for outcomes to occur?

Was follow up complete, and if not, were the reasons to loss to follow up described and explored?

Were strategies to address incomplete follow up utilized?

Was appropriate statistical analysis used?

JBI for analytical cross-sectional

Study

Were the criteria for inclusion in the sample clearly defined?

Were the study subjects and the setting described in detail?

Was the exposure measured in a valid and reliable way?

Were objective, standard criteria used for measurement of the condition?

Were confounding factors identified?

Were strategies to deal with confounding factors stated?

Were the outcomes measured in a valid and reliable way?

Was appropriate statistical analysis used?

JBI for prevalence cross-sectional

Study

Was the sample frame appropriate to address the target population?

Were study participants sampled in an appropriate way?

Was the sample size adequate?

Were the study subjects and the setting described in detail?

Was the data analysis conducted with sufficient coverage of the identified sample?

Were valid methods used for the identification of the condition?

Was the condition measured in a standard, reliable way for all participants?

Was there appropriate statistical analysis?

Was the response rate adequate, and if not, was the low response rate managed appropriately?

JBI for case series

Study

Were there clear criteria for inclusion in the case series?

Was the condition measured in a standard, reliable way for all participants included in the case series?

Were valid methods used for identification of the condition for all participants included in the case series?

Did the case series have consecutive inclusion of participants?

Did the case series have complete inclusion of participants?

Was there clear reporting of the demographics of the participants in the study?

Was there clear reporting of clinical information of the participants?

Were the outcomes or follow up results of cases clearly reported?

Was there clear reporting of the presenting site(s)/clinic(s) demographic information?

Was statistical analysis appropriate?

JBI for Case Reports/Studies

Study

Were patient’s demographic characteristics clearly described?

Was the patient’s history clearly described and presented as a timeline?

Was the current clinical condition of the patient on presentation clearly described?

Were diagnostic tests or assessment methods and the results clearly described?

Was the intervention(s) or treatment procedure(s) clearly described?

Was the post-intervention clinical condition clearly described?

Were adverse events (harms) or unanticipated events identified and described?

Does the case report provide takeaway lessons?

JBI for Qualitative

Study

Is there congruity between the stated philosophical perspective and the research methodology?

Is there congruity between the research methodology and the research question or objectives?

Is there congruity between the research methodology and the methods used to collect data?

Is there congruity between the research methodology and the representation and analysis of data?

Is there congruity between the research methodology and the interpretation of results?

Is there a statement locating the researcher culturally or theoretically?

Is the influence of the researcher on the research, and vice- versa, addressed?

Are participants, and their voices, adequately represented?

Is the research ethical according to current criteria or, for recent studies, and is there evidence of ethical approval by an appropriate body?

Do the conclusions drawn in the research report flow from the analysis, or interpretation, of the data?

Criteria for assessment for MMAT

MMAT for mixed method

Study

Are there clear research questions?

Do the collected data allow to address the research questions?

Is the qualitative approach appropriate to answer the research question?

Are the qualitative data collection methods adequate to address the research question?

Are the findings adequately derived from the data?

Is the interpretation of results sufficiently substantiated by data?

Is there coherence between qualitative data sources, collection, analysis and interpretation?

Are the participants representative of the target population?

Are measurements appropriate regarding both the outcome and exposure/intervention?

Are there complete outcome data?

Are the confounders accounted for in the design and analysis?

During the study period, is the intervention/exposure administered as intended?

Is there an adequate rationale for using a mixed methods design to address the research question?

Are the different components of the study effectively integrated to answer the research question?

Are the results adequately brought together into overall interpretations?

Are divergences and inconsistencies between quantitative and qualitative results adequately addressed?

Do the different components of the study adhere to the quality criteria of each tradition of the methods involved?