This study included five stages: systematic literature search, questionnaire development, panel formation, Delphi consensus survey rounds and guideline development.
Systematic literature search
A systematic search was conducted in September 2019 by one researcher (EY, a psychology honours student), who was supervised by two other researchers (KSB, FAC) to find statements about how a member of the public can help someone experiencing a panic attack, including how to recognise the signs of a panic attack, what to do if someone is having a panic attack, how to communicate with someone experiencing a panic attack and offer short-term assistance for its duration, and how to assist them to seek appropriate professional help if it is needed. Research publications, books and online content were included in the search for relevant statements. All searches were set to return results published since 2007, as the aim was to find content that had not been covered by the literature search in the original study. We used a combination of the original search terms from 2007 and additional terms that have become more common in the intervening timeframe, as described below.
The search engines used were: Google.com, Google.com.au, Google.co.uk, Google.nz, and Google.ca. Two searches were run in each engine: (1) ‘how to help someone having attack panic attacks AND first aid OR intervention’, (2) ‘what to do if someone has a panic attack OR panic disorder’. These terms were chosen because they delivered more relevant results than other combinations. The original 2009 study used 3 searches: (1) 'panic attack' and 'self help', (2) 'panic attack' and 'first aid', and (3) 'panic attack' and (care or carer or caring).
Searches were conducted in private or incognito mode to minimise the influence of Google’s search algorithms. The search settings were adjusted each time to reflect the country of the search engine. As with previous similar Delphi studies [30], websites returned in the top 50 results from each search were reviewed as it has been found that the quality of the resources declines after the first 50 results [31]. Any further sources linked on these websites that were thought to contain useful information were also screened. Overall, 523 websites were identified for potential first aid helping actions, duplicate sites were deleted, and relevant statements were found on 22 of these sites.
The same search terms used for the Google search engines were also used to run searches within Google Scholar (see above). In addition, PsycInfo and PubMed were each used to run two searches: (1) ‘help OR support AND panic attack’, (2) ‘panic attacks AND first aid OR intervention’. In the 2009 study, the search terms 'panic attack AND intervention OR (first aid)’ were used. Searches on these databases returned a total of 299 results. Any duplicates were deleted and the remaining articles were then screened for relevance. The irrelevant articles were excluded through a hierarchical screening process, starting with titles, abstracts and then a full-text review. Following this process, 2 articles were deemed potentially relevant. Neither of these two articles read in full contained relevant statements. This was not wholly unexpected given the few results returned from database searches in previous Delphi studies that aimed to redevelop MHFA guidelines [22, 23] and further demonstrates that there is little evidence-based guidance available for members of the public wishing to support a person experiencing a panic attack.
To locate relevant books, an advanced search of Google Books was conducted using the same search terms that were used on Google web search engines (see above). These two searches produced the most relevant results when tested. This differed from the 2009 study used that used Amazon to run a title and keyword search with the single term ‘panic’ [25]. The two searches returned 100 books, and 30 were screened. There were no new, relevant statements found in these books. See Fig. 1 for a summary of the literature search.
Questionnaire development
Strategies obtained from the systematic search on providing mental health first aid to someone experiencing a panic attack were drafted into single-idea, action-oriented statements that maintained the original source’s meaning (KSB, FAC). The first questionnaire was formed using these statements, as well as some statements from the previous 2009 Delphi questionnaire. The statements included from the previous Delphi were those that were endorsed for the original guidelines, as well as those that were endorsed by 50% or more of both the original panels. All statements were sorted into thematic categories.
A working group of researchers (KJC, AR, NJR, AFJ, BAK, CMK, AJM) reviewed all collated statements. Statements were considered acceptable for inclusion in the questionnaire if the researchers agreed that they described how a first aider could assist someone experiencing a panic attack (n = 125). The working group edited the statements to improve clarity, relevance and actionability of items, for example by re-wording statements and adding examples. They did not make judgements about the content of the statements. Examples of the types of statements included in the questionnaire include:
“The first aider should give the person some space, so that they do not feel crowded.”
“If the first aider knows that the person has had a panic attack in the past, they should ask them what has worked.”
The statements that were included from the 2009 Delphi questionnaires were reviewed in consideration of the updated definition of mental health first aid [21] and the first aider’s role, as well as for comprehensibility. Some of these items were reworded to make them clearer or to capture new actions suggested by the literature.
A final review was undertaken to eliminate any remaining repetition, maximise comprehensibility and finalise the survey. Please see Additional file 1: Round 1 survey, which contains 125 items reviewed and agreed upon by the working group.
Panel formation
Participants were recruited by KJC from high-income, western countries with developed health systems (including Australia, Canada, countries throughout Europe, Ireland, New Zealand, United Kingdom and the United States of America). Panellists were required to speak English and have professional experience in the area of panic attacks and panic disorder (e.g. as a researcher, clinician, mental health worker), or lived experience with panic attacks.
In line with the 2009 study that developed the original guidelines, the present study aimed to recruit two panels: a professional panel and a lived experience panel made up of consumers and carers. Although early Delphi studies recruited separate panels of consumers and carers, recent Delphi studies have encountered difficulty recruiting carers [22, 29]. It was therefore decided that one lived experience panel would be recruited from the outset.
A Delphi panel of at least 20 individuals has been found to yield stable results [32]. To allow for attrition over the survey rounds, this study aimed to recruit at least 30 participants to both panels. Due to challenges in recruitment these targets were not met and a decision was made by researchers to combine the lived experience group and the professional group into one panel.
Professional experts were recruited through direct editorial boards of relevant academic journals, professional bodies, mental health advocacy organisations and direct invitation, e.g. researchers and authors identified through their published work in the field. Lived experience experts were recruited through mental health advocacy organisations or direct invitation, e.g. lived experience advocates identified through their public speaking roles or authorship of websites or books. The study was also advertised through MHFA Australia’s network of Instructors, newsletter, website and social media. Experts were also asked to nominate anyone else they knew who they felt would be appropriate panel members.
All individuals who expressed interest were provided with a Plain Language Statement. Confirmed panellists were invited to the Round 1 survey via email. Participants were not reimbursed for their participation.
Delphi consensus survey rounds
The methodology used in the Delphi survey rounds, and detailed below, drew on procedures used in previous studies [22, 24, 28, 29]. Panel members completed online questionnaires, rating each statement according to how important it was that the item be included in the guidelines. A 5-point Likert scale was used (‘essential’, ‘important’, ‘don’t know/depends’, ‘unimportant’ or ‘should not be included’). Statements that achieved substantial consensus as being ‘essential’ or ‘important’ amongst the panellists were included as recommended actions for assisting someone experiencing a panic attack. Questionnaires were presented to panellists via a survey website, Survey Monkey, in three sequential rounds (see Additional files 1, 2 and 3).
The data were analysed to measure the level of consensus, with statements categorised based on the following predetermined criteria:
-
1.
Endorsed: statements rated as ‘essential’ or ‘important’ by 80% or more of the panel.
-
2.
Re-rate: statements rated as ‘essential’ or ‘important’ by 70–79.9% of the panel.
-
3.
Rejected: all other statements were excluded.
The Round 1 survey questionnaire collected demographic information and required participants to nominate their primary expertise in the area of panic attacks, as well as any additional expertise. In Round 1, panel members were also invited to provide open-ended feedback after each section of the questionnaire. This gave panellists the opportunity to suggest helping actions that were not already included. A working group of researchers reviewed this feedback for suggestions that contained original ideas. These suggestions were used to develop new helping statements to be included in the subsequent Round 2 questionnaire. Statements that attracted feedback suggesting ambiguity in the interpretation of their meaning were re-phrased to make them clearer. New statements were included in Round 2 alongside statements from Round 1 that met the criteria to be re-rated. In the Round 3 survey, participants re-rated any statements that were neither endorsed nor rejected in the previous round. Items not endorsed by consensus after being re-rated in Round 3 were rejected from inclusion in the guidelines.
After each round, panellists were sent individualised reports containing a summary of the results (KJC). All reports listed statements that had been endorsed for inclusion in the guidelines and statements that had been rejected. The Round 1 and 2 reports included a section presenting statements to be re-rated in the next survey round. These reports were tailored to include the individual panellist’s rating for each re-rate statement, alongside a summary rating of the combined panel for the statement to allow direct comparison of the panel member’s response to that of the group. Panel members could then decide whether to maintain or modify their ratings in the next survey round.
Guideline development
The statements endorsed by panellists across the three survey rounds were compiled into thematic sections to form the draft guidelines (KJC). Statements were re-written as integrated text and any repetition was deleted. A working group met to finalise structure and wording (KJC, AR, NJR, AFJ, BAK, CMK, AJM). The draft guidelines were then disseminated to panellists for any final feedback and endorsement (KJC). At this point panellists could not suggest new content; however, they were invited to make suggestions that would improve clarity and reduce ambiguity.
Ethical considerations
The University of Melbourne Human Research Ethics Committee approved this research. Participants provided informed consent by clicking ‘yes’ to a question in the Round 1 survey.