Skip to main content

Table 3 Quality appraisal of articles included in review with JBI and MMAT tools

From: Understanding the experiences of hikikomori through the lens of the CHIME framework: connectedness, hope and optimism, identity, meaning in life, and empowerment; systematic review

Criteria for assessment in JBI
JBI for quasi-experimental
Study Is it clear in the study what is the ‘cause’ and what is the ‘effect’? Were the participants included in any comparisons similar? Were the participants included in any comparisons receiving similar treatment/care, other than the exposure or intervention of interest? Was there a control group? Were there multiple measurements of the outcome both pre and post the intervention/exposure? Was follow up complete and if not, were differences between groups in terms of their follow up adequately described and analyzed? Were the outcomes of participants included in any comparisons measured in the same way? Were outcomes measured in a reliable way? Was appropriate statistical analysis used?
Lee et al. [13] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ? ✓
Malagón-Amor et al. [39] ✓ ✓ ✓ X ✓ ✓ ✓ ? ✓
Law et al. [74] ✓ ✓ ✓ ✓ ✓ ✓ X ? ✓
Yokoyama et al. [18] ✓ ✓ ✓ X ✓ ? ✓ ? N/A
Chan [75] ✓ ✓ ✓ X ✓ ? ✓ ? ✓
JBI for case control
Study Were the groups comparable other than the presence of disease in cases or the absence of disease in controls? Were the groups comparable other than the presence of disease in cases or the absence of disease in controls? Were the same criteria used for identification of cases and controls? Was exposure measured in a standard, valid and reliable way? Was exposure measured in the same way for cases and controls? Were confounding factors identified? Were strategies to deal with confounding factors stated? Were outcomes assessed in a standard, valid and reliable way for cases and controls? Was the exposure period of interest long enough to be meaningful? Was appropriate statistical analysis used?
Katsuki et al. [65] ✓ ? ✓ ✓ ✓ X X ✓ N/A ✓
JBI for analytical cohort
Study Were the two groups similar and recruited from the same population? Were the exposures measured similarly to assign people to both exposed and unexposed groups? Was the exposure measured in a valid and reliable way? Were confounding factors identified? Were strategies to deal with confounding factors stated? Were the groups/participants free of the outcome at the start of the study? Were the outcomes measured in a valid and reliable way? Was the follow up time reported and sufficient to be long enough for outcomes to occur? Was follow up complete, and if not, were the reasons to loss to follow up described and explored? Were strategies to address incomplete follow up utilized? Was appropriate statistical analysis used?
Yuen et al. [35] ✓ ✓ ✓ X X ✓ ✓ ✓ ✓ ✓ ✓
JBI for analytical cross-sectional
Study Were the criteria for inclusion in the sample clearly defined? Were the study subjects and the setting described in detail? Was the exposure measured in a valid and reliable way? Were objective, standard criteria used for measurement of the condition? Were confounding factors identified? Were strategies to deal with confounding factors stated? Were the outcomes measured in a valid and reliable way? Was appropriate statistical analysis used?
Kondo et al. [40] ✓ ✓ ✓ ✓ X X ✓ ✓
Krieg and Dickie [37] ✓ ✓ ✓ ✓ X X ✓ ✓
Nagata et al. [64] ✓ ✓ ✓ ✓ X X ✓ ✓
Chan and Lo [4] ✓ ✓ ✓ ✓ X X ✓ ✓
Uchida and Norasakkunkit [58] ✓ ✓ ✓ ✓ X X ✓ ✓
Umeda et al. [63] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Yuen et al. [41] ✓ ✓ ✓ ✓ X X ✓ ✓
Yong and Nomura [43] ✓ ✓ ✓ ✓ X X ✓ ✓
Wu et al. [42] ✓ ✓ ✓ ✓ ? ? ✓ ✓
JBI for prevalence cross-sectional
Study Was the sample frame appropriate to address the target population? Were study participants sampled in an appropriate way? Was the sample size adequate? Were the study subjects and the setting described in detail? Was the data analysis conducted with sufficient coverage of the identified sample? Were valid methods used for the identification of the condition? Was the condition measured in a standard, reliable way for all participants? Was there appropriate statistical analysis? Was the response rate adequate, and if not, was the low response rate managed appropriately?
Koyama et al. [62] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Malagón-Amor et al. [59] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Chauliac et al. [38] ✓ ✓ N/A ✓ ✓ ✓ ✓ ✓ N/A
Frankova [61] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
JBI for case series
Study Were there clear criteria for inclusion in the case series? Was the condition measured in a standard, reliable way for all participants included in the case series? Were valid methods used for identification of the condition for all participants included in the case series? Did the case series have consecutive inclusion of participants? Did the case series have complete inclusion of participants? Was there clear reporting of the demographics of the participants in the study? Was there clear reporting of clinical information of the participants? Were the outcomes or follow up results of cases clearly reported? Was there clear reporting of the presenting site(s)/clinic(s) demographic information? Was statistical analysis appropriate?
Teo et al. [36] ✓ ✓ ✓ ✓ ✓ ✓ ? ✓ ✓ ✓
JBI for Case Reports/Studies
Study Were patient’s demographic characteristics clearly described? Was the patient’s history clearly described and presented as a timeline? Was the current clinical condition of the patient on presentation clearly described? Were diagnostic tests or assessment methods and the results clearly described? Was the intervention(s) or treatment procedure(s) clearly described? Was the post-intervention clinical condition clearly described? Were adverse events (harms) or unanticipated events identified and described? Does the case report provide takeaway lessons?
Sakamoto et al. [47] ✓ ✓ ✓ ✓ ? ✓ ✓ ✓
Hattori [73] ✓ ✓ ✓ ✓ ✓ ✓ X ✓
Suwa and Suzuki [51] ✓ ✓ ✓ X X ✓ ✓ X
Teo [49] ? ✓ ✓ ✓ ✓ ✓ X ✓
Overjero et al. (2014) X ✓ ✓ ✓ ✓ ✓ X ✓
Ranieri [46] ✓ ✓ ✓ X ? ✓ X ✓
Kato et al. [44] X ✓ ✓ ✓ ? ✓ ✓ X
Ranieri [46] ? ✓ ✓ ? ? ✓ ✓ ✓
Matsuguma et al. [77] ✓ ✓ ✓ ✓ ? ✓ X ✓
Silić et al. (2019) ✓ ✓ ✓ ✓ ✓ ✓ X ✓
Roza et al. [72] X ✓ ✓ ✓ ✓ ✓ X ✓
JBI for Qualitative
Study Is there congruity between the stated philosophical perspective and the research methodology? Is there congruity between the research methodology and the research question or objectives? Is there congruity between the research methodology and the methods used to collect data? Is there congruity between the research methodology and the representation and analysis of data? Is there congruity between the research methodology and the interpretation of results? Is there a statement locating the researcher culturally or theoretically? Is the influence of the researcher on the research, and vice- versa, addressed? Are participants, and their voices, adequately represented? Is the research ethical according to current criteria or, for recent studies, and is there evidence of ethical approval by an appropriate body? Do the conclusions drawn in the research report flow from the analysis, or interpretation, of the data?
Ogino [56] ✓ ✓ ✓ ✓ ✓ ✓ ? ✓ X ✓
Kaneko [52] ✓ ✓ ✓ ✓ ✓ ✓ ? ✓ ? ✓
Wong and Ying [54] ✓ ✓ ✓ ✓ ✓ ✓ ? ✓ ? ✓
Wong [6] ✓ ✓ ✓ ✓ ✓ ✓ ? ? ? ✓
Wong [70] ✓ ✓ ✓ ✓ ✓ ✓ ? ✓ ✓ ✓
Tajan [11] ✓ ✓ ✓ ✓ ✓ ✓ ? ✓ ? ✓
Rubinstein [71] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ? ✓ ✓
Yong and Kaneko [57] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Li and Wong [53] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ? ✓
Criteria for assessment for MMAT
MMAT for mixed method
Study Are there clear research questions? Do the collected data allow to address the research questions? Is the qualitative approach appropriate to answer the research question? Are the qualitative data collection methods adequate to address the research question? Are the findings adequately derived from the data? Is the interpretation of results sufficiently substantiated by data? Is there coherence between qualitative data sources, collection, analysis and interpretation? Are the participants representative of the target population? Are measurements appropriate regarding both the outcome and exposure/intervention? Are there complete outcome data? Are the confounders accounted for in the design and analysis? During the study period, is the intervention/exposure administered as intended? Is there an adequate rationale for using a mixed methods design to address the research question? Are the different components of the study effectively integrated to answer the research question? Are the results adequately brought together into overall interpretations? Are divergences and inconsistencies between quantitative and qualitative results adequately addressed? Do the different components of the study adhere to the quality criteria of each tradition of the methods involved?
Chan and Lo [60] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ X ✓ ✓ ✓ ✓ ✓ ✓
Chan [76] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ X ✓ ✓ ✓ ✓ ✓ ✓
Chan [55] ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ X ✓ ✓ ✓ ✓ ✓ ✓
  1. ✓, Yes; X, No; Question Mark (?) , Unclear or Can’t tell; N/A, Not applicable; JBI, Joanne Brigg’s Institute Appraisal Tool; MMAT, Mixed Method Appraisal Tool.
Back to article page

BMC Psychology

ISSN: 2050-7283

Contact us