Advertisement
Clinical| Volume 49, ISSUE 3, P371-386.e5, May 2023

Download started.

Ok

A Systematic Review of Violence Risk Assessment Tools Currently Used in Emergency Care Settings

Open AccessPublished:December 28, 2022DOI:https://doi.org/10.1016/j.jen.2022.11.006

      Abstract

      Introduction

      Violence risk assessment is commonplace in mental health settings and is gradually being used in emergency care. The aim of this review was to explore the efficacy of undertaking violence risk assessment in reducing patient violence and to identify which tool(s), if any, are best placed to do so.

      Methods

      CINAHL, Embase, Medline, and Web of Science database searches were supplemented with a search of Google Scholar. Risk of bias assessments were made for intervention studies, and the quality of tool development/testing studies was assessed against scale development criteria. Narrative synthesis was undertaken.

      Results

      Eight studies were included. Three existing violence risk assessment tools featured across the studies, all of which were developed for use with mental health patients. Three newly developed tools were developed for emergency care, and 1 additional tool was an adaptation of an extant tool. Where tested, the tools demonstrated that they were able to predict patient violence, but did not reduce restraint use. The quality issues of the studies are a significant limitation and highlight the need for additional research in this area.

      Discussion

      There is a paucity of high-quality evidence evaluating the psychometric properties of violence risk assessment tools currently used along the emergency care pathway. Multiple tools exist, and they could have a role in reducing violence in emergency care. However, the limited testing of their psychometric properties, acceptability, feasibility, and usability in emergency care means that it is not possible to favor one tool over another until further research is conducted.

      Graphical abstract

      Key words

      Contribution to Emergency Nursing Practice
      • Workplace violence is common in emergency care settings and has negative consequences for patients, staff, and services. Structured violence risk assessment is commonplace in mental health settings and is gradually becoming more accepted within emergency care.
      • This review has found that violence risk assessment tools may be feasible for use in emergency department. There is currently, however, insufficient high-quality evidence to draw conclusions about the predictive capability of these tools in emergency care settings.
      • Violence risk assessment can identify patients in emergency care who are at risk of becoming violent, but the evidence to support choosing one tool over another is not yet available. Further research using these tools in emergency settings is needed before evidence-based recommendations can be made.

      Introduction

      Globally, staff working in emergency care settings experience violence from patients and visitors at a disproportionate rate. A recent international systematic review and meta-analysis
      found that emergency departments had the highest 12-month prevalence of violence across all hospital settings. The same review found that nurses had the highest exposure to violence across occupational groups. For the purposes of our study, we use the term violence to describe any nonverbal, verbal, or physical behavior exhibited by a person that makes it difficult to deliver good care safely. Staff working in emergency department appear resigned to the inevitability of experiencing such violence.
      • Ashton R.A.
      • Morris L.
      • Smith I.
      A qualitative meta-synthesis of emergency department staff experiences of violence and aggression.
      Workplace violence has wide-ranging detrimental consequences.
      • Pich J.V.
      • Kable A.
      • Hazelton M.
      Antecedents and precipitants of patient-related violence in the emergency department: results from the Australian VENT Study (Violence in Emergency Nursing and Triage).
      Staff absence because of the physical or emotional effects of workplace violence has significant financial implications.
      It is estimated that 2% of staff are lost as a consequence of workplace violence, leading to significant recruitment costs.
      Violence also causes disruptions to patient care, with nurses losing concentration and working at reduced efficiency
      • Hassankhani H.
      • Parizad N.
      • Gacki-Smith J.
      • Rahmani A.
      • Mohammadi E.
      The consequences of violence against nurses working in the emergency department: a qualitative study.
      and functioning at a heightened level of anxiety.
      • Monks R.
      • Topping A.
      • Newell R.
      The dissonant care management of illicit drug users in medical wards, the views of nurses and patients: a grounded theory study.
      Violence also is associated with task delays and medication errors.
      • Roche M.
      • Diers D.
      • Duffield C.
      • Catling-Paull C.
      Violence toward nurses, the work environment, and patient outcomes.
      Several structured tools have been developed to aid risk assessment of imminent violence, most commonly in mental health settings, but they are being used increasingly in other areas.
      • D’Ettorre G.
      • Pellicani V.
      • Mazzotta M.
      • Vullo A.
      Preventing and managing workplace violence against healthcare workers in emergency departments.
      • Sharifi S.
      • Shahoei R.
      • Nouri B.
      • Almvik R.
      • Valiee S.
      Effect of an education program, risk assessment checklist and prevention protocol on violence against emergency department nurses: a single center before and after study.
      • Kim S.C.
      • Ideker K.
      • Todicheeney-Mannes D.
      Usefulness of Aggressive Behaviour Risk Assessment Tool for prospectively identifying violent patients in medical and surgical units.
      A recent scoping review by Cabilan and Johnston
      • Cabilan C.J.
      • Johnston A.N.
      Review article: identifying occupational violence patient risk factors and risk assessment tools in the emergency department: a scoping review.
      identified 5 violence risk assessment tools with a history of use in ED settings; however, the review reported that 3 lacked any evidence of predictive validity. In fact, of the 5 tools identified, only 1, the Brøset Violence Checklist (BVC),
      • Woods P.
      • Almvik R.
      The Brøset Violence Checklist (BVC).
      was intended for use as a risk assessment prediction tool rather than an aide memoire and was the only one whose psychometric properties were evaluated in an emergency care setting. The BVC was developed, and has been used with some success, to predict violence in mental health settings.
      • Dickens G.L.
      • O’Shea L.E.
      • Christensen M.
      Structured assessments for imminent aggression in mental health and correctional settings: systematic review and meta-analysis: risk assessment for imminent violence.
      With evidence that violence risk assessment tools are gradually finding their way into emergency care,
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      it is important not only to identify those that have been implemented but also to establish which tools are practical and effective. Therefore, we aimed to examine the psychometric properties, acceptability, feasibility, and usability of violence risk assessment tools that have been evaluated in emergency care. For the purposes of this review, the constructs of acceptability, feasibility, and usability will be interpreted broadly, respectively, relating to factors affecting users’ willingness to adopt interventions, individual or structural factors affecting the extent to which interventions can be implemented effectively, and factors pertaining to the user experience.
      • Ginsburg A.S.
      • Tawiah Agyemang C.
      • Ambler G.
      • et al.
      mPneumonia, an innovation for diagnosing and treating childhood pneumonia in low-resource settings: a feasibility, usability and acceptability study in Ghana.
      In doing so, we aimed to explore the efficacy of undertaking violence risk assessment in predicting and reducing patient violence and to identify which tool(s), if any, are best placed to do so.

      Methods

      Design

      We undertook a systematic review; our reporting follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.
      • Page M.J.
      • McKenzie J.E.
      • Bossuyt P.M.
      • et al.
      The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
      The protocol for this review was registered at the International Prospective Register of Ongoing Systematic Reviews (CRD42021285461). The protocol was registered as a rapid review, but during conduct of the review, the team agreed that a full systematic review was preferable and achievable within existing resources.

      Eligibility Criteria

      Eligible studies were (1) primary research; (2) published in peer-reviewed journals; (3) in English language; (4) published since 2007 (the earliest publication date of the tools identified by Cabilan and Johnston
      • Cabilan C.J.
      • Johnston A.N.
      Review article: identifying occupational violence patient risk factors and risk assessment tools in the emergency department: a scoping review.
      ); (5) evaluations of the psychometric properties, acceptability, feasibility, or usability of violence risk assessment tools; and (6) focused on emergency care pathways (emergency department and acute medical units [AMUs] or equivalent: for example, admission areas for acute medical patients with a length of stay up to 48 hours). Studies within specialist emergency care pathways (eg, pediatric, psychiatric) were excluded. For the purposes of our review, “violence” refers to both actual and threatened physical acts or verbal abuse perpetrated by emergency attendees (patients or their relatives/friends/companions) against others or objects.
      As the broad constructs of feasibility, usability, and acceptability can be captured by both quantitative and qualitative data, we did not exclude any primary research studies based on methodological approach alone.

      Search Strategy

      A study by Bramer et al
      • Bramer W.M.
      • Rethlefsen M.L.
      • Kleijnen J.
      • Franco O.H.
      Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study.
      found that optimal searches in systematic reviews should include the following databases: Embase, Medline, Web of Science, and Google Scholar. Accordingly, we used these 4 databases for our searches and added Cumulative Index to Nursing and Allied Health Literature Plus to ensure that we captured relevant nursing literature. Owing to the limited search functionality of Google Scholar, we only screened the first 200 references identified by this database, ranked by relevance.
      • Bramer W.M.
      • Rethlefsen M.L.
      • Kleijnen J.
      • Franco O.H.
      Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study.
      Our search strategy was based on Cabilan and Johnston’s
      • Cabilan C.J.
      • Johnston A.N.
      Review article: identifying occupational violence patient risk factors and risk assessment tools in the emergency department: a scoping review.
      strategy but was amended to capture literature related to our broader conceptualization of the emergency care pathway and to the relevant properties of tools identified. Our search terms were mapped according to the population or problem, intervention, comparison, outcomes, context framework (Table 1), see Supplementary Tables 1-4 for full search terms.
      Table 1Population or problem, intervention, comparison, outcomes, context framework
      CriterionDescription
      Population or problemViolence toward others, perpetrated by emergency care attendees
      InterventionStructured risk assessment tools
      ComparisonNot applicable
      OutcomesPsychometric properties (including validity, reliability, internal consistency and predictive validity), feasibility, usability, and acceptability
      ContextEmergency care pathways
      Searches were undertaken in October 2021 and supplemented by regular ongoing searches for keyword terms via Google Scholar until July 2022. In addition, the authors of any relevant articles that were not published in peer-reviewed journals (eg, dissertations) were contacted to ensure that we did not miss any work they might have published. Screening by title and abstract was undertaken independently by 2 reviewers (D.S. and N.H.), with 1 reviewer (D.S.) then completing full-text screening. The shortlist of papers possibly eligible for inclusion was screened by a third reviewer (L.L.D.). Forward and backward chain searching was conducted on all eligible papers.

      Risk of Bias and Quality Assessment

      All intervention studies were assessed for risk of bias using the Risk of Bias in Non-randomized Studies of Interventions tool.
      • Sterne J.A.C.
      • Hernán M.A.
      • Reeves B.C.
      • et al.
      ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions.
      The studies that described tool development/testing were assessed against scale development criteria described by Boateng et al
      • Boateng G.O.
      • Neilands T.B.
      • Frongillo E.A.
      • Melgar-Quiñonez H.R.
      • Young S.L.
      Best practices for developing and validating scales for health, social, and behavioral research: a primer.
      ; criteria relating to factors and dimensionality were removed as these were not relevant to the development of risk assessment tools. Quality assessment of included studies was undertaken by D.S. and N.H. and checked by L.L.D. and G.D.

      Data Extraction and Synthesis

      Data were extracted by D.S. and checked independently by N.H. As presented in our protocol, predefined subheadings were amended and/or discarded as appropriate. These decisions were initially made by D.S. and later discussed with the whole team until consensus was reached.
      Because of methodological and clinical heterogeneity in the included studies, we were unable to undertake a statistical meta-analysis; therefore, narrative synthesis was undertaken. Statistical information about predictive efficacy, interrater reliability, and intervention efficacy were extracted. Predictive efficacy data included sensitivity and specificity (true positive and true negative cases as proportions of all positive and negative predictions, respectively), positive predictive validity (odds of those predicted to be violent who actually went on to be violent), area under the receiver operating characteristic curve (AUC; a summary statistic [range 0-1] of a tool’s overall ability to discriminate between positive and negative cases; interpretation AUC = 0.5 equivalent to chance, 0.7-0.79 acceptable, 0.8-0.89 excellent, 9.0-1.0 outstanding), and odds ratios (the odds that an individual who is violent was assessed as at increased risk of violence compared with the odds that a nonviolent individual was assessed as not at increased risk of violence). Information was extracted for all cut-off points reported. Information about interrater reliability involved kappa, a measure of agreement between independent raters: 0.40 to 0.59 = weak agreement, 0.60 to 0.79 = moderate agreement, 0.80 to 0.90 = strong agreement, and above 0.90 is almost perfect.
      • McHugh M.L.
      Interrater reliability: the kappa statistic.
      Information about intervention efficacy included P values indicating statistical significance and relative risk for all outcomes reported. Data about the feasibility and usability of tools were extracted where available.

      Results

      Search Outcome

      As a result of the search strategy, 8 studies were deemed eligible for inclusion (Figure).
      Figure thumbnail gr1
      FigurePreferred reporting items for systematic reviews and meta-analyses flow diagram.18 CINAHL, Cumulative Index to Nursing and Allied Health Literature.

      Summary of Included Studies

      Of the 8 included studies, 2 used cohort designs, of which 1 was retrospective
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      and 1 prospective
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      ; 2 used quality improvement designs
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      ,
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      ; 1 used a before-and-after design
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      ; 1 used tool development methods
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      ; 1 tested a tool
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      ; and 1 used nonparticipant observation.
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      Four studies were deemed intervention studies, with various outcomes,
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      whereas 4 aimed to test/develop tools.
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      ,
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      Seven studies were conducted entirely in emergency departments, and 129 included observations of which 82.4% of the observations were conducted in the emergency department. No studies took place in AMUs or equivalent. Four studies were conducted in Australia
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      ,
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      ,
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      ,
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      and 4 in the United States.
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      ,
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.

      Violence Risk Assessment Tools

      Three of the studies described the development and testing of new risk assessment tools.
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      ,
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      ,
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      These were all created for use within emergency care pathways. One was created using extant literature and expert opinion (Queensland Occupational Violence Patient Risk Assessment tOol [QOVPRAO])
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      ; 1 supplemented this approach with chart audits, (Emergent Documentation Aggression Rating Tool [EDART])
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      ; and 1 used nonparticipant observation (Violence Assessment Tool [VAT]).
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      Four studies tested existing tools: the Behavioral Activity Rating Scale (BARS),
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      ,
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      the BVC,
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      and the Dynamic Appraisal of Situational Aggression (DASA)
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      (Table 2). The final study combined the BVC with a response framework for use in the emergency department to create the behaviors of concern (BOC) chart.
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      All of the existing tools were originally developed either for use in mental health settings (BVC, DASA) or for use with patients with psychosis (BARS).
      Table 2Risk assessment tools
      ToolIncluded studies; developed by (if different)Development setting/countryContentScoringInterpretationRisk management
      Behavioral Activity Rating ScaleLegambi et al,
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      Schumacher et al
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      ;

      Swift et al
      • Swift R.H.
      • Harrigan E.P.
      • Cappelleri J.C.
      • Kramer D.
      • Chandler L.P.
      Validation of the Behavioural Activity Rating Scale (BARS): a novel measure of activity in agitated patients.
      Setting not stated (developed to evaluate the effect of psychotropic medication on agitated behavior in patients experiencing psychosis), United StatesSingle-item question consisting of 7 categories: 1 = difficult or unable to rouse; 2 = asleep, but responds normally to verbal or physical contact; 3 = drowsy, appears sedated; 4 = quiet and awake (normal level of activity); 5 = signs of overt (physical or verbal) activity, calms down with instruction; 6 = extremely or continuously active, not requiring restraint; 7 = violent, requires restraint1-71-4 = nonresponsive/no agitation

      5-7 = increasing severity of agitation
      None identified
      BVC/BOCBVC: Partridge and Affleck
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      ; Almvik and Woods
      • Woods P.
      • Almvik R.
      The Brøset Violence Checklist (BVC).


      BOC: Senz et al
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      BVC: secure mental health, Norway

      BOC: additional management matrix developed in emergency department, Australia
      Six items:

      - confusion

      - irritability

      - boisterousness

      - physical threats

      - verbal threats

      - attacking objects
      Each item scored 0 (absent) or 1 (present)BVC

      0 = low risk

      1-2 = moderate risk

      ≥3 = high risk

      BOC

      0 = low risk

      1 = moderate risk

      ≥2 = high risk
      BVC: None identified

      BOC: interventions identified for each level of risk by: general, nursing, medical, security
      Dynamic Appraisal of Situational AggressionConnor et al
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      ;

      Ogloff and Daffern
      • Ogloff J.R.
      • Daffern M.
      The dynamic appraisal of situational aggression: an instrument to assess risk for imminent aggression in psychiatric inpatients.
      Secure mental health, AustraliaSeven items:

      -irritability

      -impulsivity

      -unwillingness to follow directions

      -sensitivity to perceived provocation

      -easily angered

      -negative attitudes

      -verbal threats
      Each item scored 0 (normal for patient) or 1 (increase in described behavior)0-1 = low risk

      2-3 = moderate risk

      >3 = high risk
      None identified
      Emergent Documentation Aggression Rating toolCampbell et al
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      Emergency department, United StatesSingle-item chart listing 6 behavior levels ranging from “no signs of aggression” to “danger to self and others” (multiple behaviors listed within each level)0-50 = no signs of aggression

      1 = early indicators

      2-5 = increasing severity
      Interventions identified for each level of aggression
      Queensland Occupational Violence Patient Risk Assessment toolCabilan et al
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      Emergency department, AustraliaThree items:

      - Aggression history

      - Behavioral concerns

      - Clinical presentation
      0 (absent)

      1 (present/yes)
      0 = low risk

      1 = moderate risk

      2-3 = high risk
      None identified
      Violence Assessment toolJackson et al
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      Acute hospital, AustraliaEighteen behavioral cues:

      - Threat of harm

      - Aggressive statements or threats

      - Intimidation

      - Clenched fists

      - Resisting care

      - Prolonged or intense glaring

      - Name calling

      - Yelling

      - Increase in volume of speech

      - Irritability

      - Pacing near nurses’ area

      - Pacing in confined areas

      - Sharp or caustic retorts

      - Demeaning inflection

      - Belligerence

      - Demanding attention

      - Humiliating remarks

      - Mumbling
      Not statedNot statedNone identified
      BOC, behaviors of concern; BVC, Brøset Violence Checklist.

      Quality of Included Studies

      Four studies were assessed for risk of bias,
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      and all were deemed at serious risk (Table 3). Although no studies were excluded based on quality, we were unable to include data from 2 studies in our syntheses of predictive efficacy, validity, and reliability owing to serious risk of confounding. Schumacher et al
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      measured the predictive validity of the BARS in relation to administration of behavioral management (ie, sedation or physical restraint). However, these interventions were prescribed by medical staff on the basis of BARS scores, thus ensuring a circular relationship where the outcome was inevitable if the predictor was positive. A similar confounder was noted in the quality improvement project described by Legambi et al,
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      where preassessment and postassessment data were collected on restraint use. The BARS was incorporated into the electronic health record, which automatically prompted staff to apply restraints on patients who scored 7 (violent). Although all studies were at low risk of bias in classification of interventions because risk assessment was routinely recorded, they were all at moderate to serious risk of bias owing to deviation from intended intervention. The 2 studies at moderate risk either did not provide adequate information on how nurses decided to undertake risk assessment
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      or only assessed patients once rather than at regular intervals.
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      The other 2 studies had more serious issues. Campbell et al
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      did not report whether restrained patients had been risk-assessed. Risk assessment occurred before the intervention as reported by Senz et al
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      as well as after, but no detail was provided about differences in how risk assessment occurred pre- or post-test.
      Table 3Risk of bias table
      AuthorsBias due to confoundingBias in selection of participantsBias in classification of interventionsBias due to deviations from intended interventionsBias due to missing dataBias in measurement of outcomesBias in selection of the reported resultsOverall assessment
      Schumacher et al
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      SeriousSeriousLowSeriousModerateModerateLowSerious risk of bias
      Campbell et al
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      ModerateModerateLowSeriousModerateModerateLowSerious risk of bias
      Legambi et al
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      SeriousSeriousLowModerateSeriousModerateLowSerious risk of bias
      Senz et al
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      ModerateModerateLowSeriousModerateLowLowSerious risk of bias
      Two studies detailed tool development,
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      ,
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      and 2 tested pre-existing tools
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      ,
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      (Table 4). Items for the newly developed tools were generated within emergency settings, through observation
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      and from the literature,
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      whereas items for the preexisting tools were generated in mental health settings.
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      ,
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      Similarly, content validity and pretesting of questions occurred in mental health settings for the preexisting tools,
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      ,
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      thus raising some concerns as neither tool was tested for these within the emergency care context. Researchers administered the tools in the development studies through observations
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      and from electronic records.
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      Table 4Critical appraisal of tool development studies table
      AuthorsItem generationContent validityPretesting of questionsAdministrationSample sizePredictive validity testingInterrater reliability
      Jackson et al
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      GoodGoodGoodSome concernsSome concernsSome concernsPoor
      Partridge and Affleck
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      Some concernsSome concernsSome concernsSome concernsGoodGoodSome concerns
      Connor et al
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      Some concernsSome concernsSome concernsGoodGoodGoodSome concerns
      Cabilan et al
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      Some concernsGoodPoorPoorGoodGoodGood

      Data Synthesis

      Studies were grouped by risk assessment tool; however, only 2 tools featured in more than 1 study (the BARS and the BVC). The psychometric properties of the tools, where available, are presented in Table 5.
      Table 5Properties of risk assessment tools
      Tool; included studiesOutcomeCut-offPredictive efficacyContent validityReliabilityIntervention efficacy
      BARS; Legambi et al
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      Restraint use----1. No statistically significant difference in restraint use following implementation (χ2 = 0.72, P = .40)
      BOC; Senz et al
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      Planned and emergency security responses (code gray); mechanical restraint----1. Reduction in planned Code Grays (RR 2.22) and emergency Code Grays (RR 0.75, absolute

      risk reduction 0.18%).

      2. No reduction in mechanical restraint use.
      BVC;

      Partridge and Affleck
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      Violence1OR 11.6Not assessed in emergency care, only in mental health settings--
      2OR 30.3
      3OR 71.4
      ≥1PPV 16.7%
      ≥2PPV 34.3%
      ≥3PPV 55.2%
      3Sens. 45.7%

      Spec. 99.4%
      DASA; Connor et al
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      Violent or aggressive behaviorScore: 1+ vs 0PPV 23% vs 5%Not assessed in emergency care, only in mental health settings--
      AUC 0.77
      EDART; Campbell et al
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      Restraint use----1. No statistically significant difference in restraint use before and after implementation (logistic interrupted time series model with time F = 2.01, P = .13)
      QOVPRAO; Cabilan et al
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      Occupational violenceAggression historyOR 9.0

      Sens. 22%

      Spec. 98%
      I-CVI 0.86K 0.60-0.75-
      BehavioralOR 13.6

      Sens. 31%

      Spec. 98%
      I-CVI 0.95
      ClinicalOR 7.1

      Sens. 55%

      Spec. 92%
      I-CVI 0.89
      Risk rating 0, 1, 2+AUC 0.77-
      Moderate riskSens. 61%

      Spec. 91%
      High riskSens. 37%

      Spec. 97%
      VAT; Jackson et al
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      ViolenceResisting health careOR 11---
      Aggressive statementsOR 7.16-
      YellingOR 6.79-
      Abusive languageOR 5.98-
      AUC, area under the curve; BARS, Behavioral Activity Rating Scale; BOC, behaviors of concern; BVC, Brøset Violence Checklist; DASA, Dynamic Appraisal of Situational Aggression; EDART, Emergent Documentation Aggression Rating Tool; I-CVI, item-level content validity index; OR, odds ratio; PPV, positive predictive value; QOVPRAO, Queensland Occupational Violence Patient Risk Assessment Tool; RR, relative risk; Sens., sensitive; Spec., specificity; VAT, Violence Assessment Tool.

      BARS

      Legambi et al
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      examined restraint use before and after implementation of the BARS and found a nonsignificant difference. During the final weeks of BARS implementation, they administered the System Usability Scale (SUS) to emergency nurses. From 30 (31% response rate) responses, the BARS received a high SUS score (83.46; SD = 11.73), indicating good usability (citing Usability.gov, the authors note that SUS scores greater than 68 indicate good usability, even with a small sample size). However, only 13 (43%) reported feeling as though the BARS helped them to better detect and manage behavioral health patients (the primary target group requiring BARS assessment in the study emergency department). In their review of patient records, Schumacher et al
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      found that only 46% of patients with a psychiatric complaint received a BARS rating at triage, indicating low adoption of the tool.

      BVC/BOC

      Partridge and Affleck
      • Partridge B.
      • Affleck J.
      Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
      calculated positive likelihood ratios (odds ratios) for the BVC using cut-off scores of 1, 2, and 3. Their findings showed that violent patients were 71.4 times more likely to have a score of ≥3 than nonviolent patients; they were 30.3 times more likely to have a score of ≥2 and 11.6 times more likely to have a score of ≥1. The study found a predictive value of 16.7% for scores ≥1, 34.3% for scores ≥2, and 55.2% for scores ≥3. This means that more than half the patients who scored 3 or more would go on to exhibit violent behaviors. When using 3 as a cut-off for BVC scores to indicate high risk of violence, sensitivity was 45.7%, and specificity was 99.4%, meaning that just under half of all violent patients and nearly all nonviolent patients were identified by the BVC.
      Before implementation of the BOC, violence risk assessment was documented 30% of the time; after implementation, this increased to 82%.
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      Furthermore, before implementation, violence risk assessment was documented 54% of the time for patients with a mental health or drug and alcohol presentation, increasing to 100% after implementation. Senz et al
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      did not assess usability of the BOC; however, they explored nurses’ confidence and abilities in a before-and-after survey. Despite statistically significant improvements in confidence to perform risk screening, there was no change in perceived ability to prevent violence.

      DASA

      Connor et al
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      calculated positive and negative predictive values for the DASA, comparing scores of ≥1 with scores of 0. They found that 23% of patients with a score of ≥1 would go on to be violent, and 95% of patients with a score of 0 would not exhibit violent behaviors. The summary AUC score of 0.79 fell in the “acceptable” category.

      EDART

      Campbell et al
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      found no statistically significant difference in restraint use before and after implementation of the EDART as assessed by a logistic interrupted time series model with time F = 2.01, P = .13.
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      To explore the usability of the EDART, a survey was administered to emergency nurses 3 months into the study’s implementation phase, receiving responses from 30 participants (62.5% response rate). Feedback about the EDART was overwhelmingly positive, with all respondents agreeing that the tool was easy to use and 28 of 30 reporting that the tool increased their ability to offer early interventions.

      QOVPRAO

      In the development of the QOVPRAO, Cabilan et al
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      found that of the 34 risk items forwarded to end users for relevance rating, 5 achieved a relevant item-level content validity index (I-CVI) (≥0.78), with consensus moderation used to direct the inclusion of additional risk items (despite achieving I-CVI scores below the 0.78 threshold). However, in a second round of content validity to rate the relevance of each of the tool’s 3 risk domains, all 3 achieved I-CVIs above the 0.78 threshold. Sensitivity for the QOVPRAO domains ranged from 22% for aggression history to 55% for concerns with clinical presentation; specificity was high for all (92%-98%). The AUC using risk rating of low (no risk domains present), moderate (1 risk domain present), and high (≥2 risk domains present) for the QOVPRAO indicated acceptable predictive validity (AUC = 0.77). Testing interrater reliability between a trained and an untrained assessor, the analysis revealed kappa values ranging from 0.60 to 0.75 for the tool’s 3 domains (P < .01), indicating moderate agreement.
      • McHugh M.L.
      Interrater reliability: the kappa statistic.

      VAT

      Jackson et al
      • Jackson D.
      • Wilkes L.
      • Luck L.
      Cues that predict violence in the hospital setting: findings from an observational study.
      examined the association between the 18 behavioral cues in the VAT and subsequent violence. Patients who resisted health care were 11 times more likely to exhibit violent behaviors than those who did not; those who made aggressive statements were 7.2 times more likely; those who yelled were 6.8 times more likely; and those who used abusive language were 6.0 times more likely.

      Discussion

      This review identified 8 studies that evaluated the psychometric properties of 7 violence risk assessment tools in emergency departments. The tools were either originally developed in mental health settings or specifically for ED settings. Only 2 tools, the BARS and the BVC, featured in more than 1 study, limiting our ability to pool results. Our findings also are limited by the quality of the included studies, with some suffering from significant methodological flaws such as unmeasured confounding variables and deviations from the intended intervention(s). However, our review addresses an important gap in the literature. The paucity of evidence about these tools’ performance in emergency settings stands in contrast to the significant body of literature on violent risk assessment in psychiatric settings,
      • Dickens G.L.
      • O’Shea L.E.
      • Christensen M.
      Structured assessments for imminent aggression in mental health and correctional settings: systematic review and meta-analysis: risk assessment for imminent violence.
      despite the similarities in violence prevalence across these settings.
      • Spector P.E.
      • Zhou Z.E.
      • Che X.X.
      Nurse exposure to physical and nonphysical violence, bullying, and sexual harassment: a quantitative review.
      Only 2 studies examined predictive validity, 1 each of the DASA and the QOVPRAO,
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      ,
      • Connor M.
      • Armbruster M.
      • Hurley K.
      • Lee E.
      • Chen B.
      • Doering L.
      Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
      with both tools demonstrating moderate performance. In studies of the DASA in mental health settings, results have ranged from acceptable to outstanding,
      • Griffith J.J.
      • Daffern M.
      • Godber T.
      Examination of the predictive validity of the dynamic appraisal of situational aggression in two mental health units.
      • Dumais A.
      • Larue C.
      • Michaud C.
      • Goulet M.H.
      Predictive validity and psychiatric nursing Staff’s perception of the clinical usefulness of the French version of the dynamic appraisal of situational aggression.
      • Lantta T.
      • Kontio R.
      • Daffern M.
      • Adams C.
      • Välimäki M.
      Using the dynamic appraisal of situational aggression with mental health inpatients: a feasibility study.
      reflecting similar findings to the 2 studies in this review. However, the clinical context should be factored into any comparisons drawn with findings from ED settings. Violence risk assessment does not occur in a vacuum. In psychiatric inpatient settings, where the DASA and BVC have seen most use and evaluation, patients are risk-assessed repeatedly throughout an inpatient stay, which will typically be much longer than in emergency care settings. Clinicians’ familiarity with patients is likely to factor into their interpretation of patient behaviors and characteristics,
      • Trenoweth S.
      Perceiving risk in dangerous situations: risks of violence among mental health inpatients.
      and the nature of violent incidents also may differ across these very different clinical contexts.
      • Llor-Esteban B.
      • Sánchez-Muñoz M.
      • Ruiz-Hernández J.A.
      • Jiménez-Barbero J.A.
      User violence towards nursing professionals in mental health services and emergency units.
      This underpins the importance of evaluating tools in the settings where they will be implemented, particularly as clinician expertise, preferences, and needs also will differ.
      Clinical approaches to risk assessment, which involve unstructured clinical judgment, are largely subjective and reliant on the assessor’s expertise, whereas actuarial approaches aim to eliminate bias by standardizing all aspects of the assessment. In mental health settings, this polarity has been somewhat addressed by the introduction of structured professional judgment approaches, which combine ratings of empirically derived risk factors together with consideration of idiosyncratic individual factors, eg, Short Term Assessment of Risk and Treatability.
      • Webster C.D.
      • Martin M.
      • Brink J.
      • Nicholls T.L.
      • Desmarais S.L.
      Manual for the Short Term Assessment of Risk and Treatability (START).
      Consideration could be given to the development of such approaches in the emergency department.
      The tools included in this review all use an actuarial approach, although, as Doyle and Dolan
      • Doyle M.
      • Dolan M.
      Violence risk assessment: combining actuarial and clinical information to structure clinical judgements for the formulation and management of risk.
      note, all risk assessment involves a degree of subjectivity. Only 1 study
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      evaluated interrater reliability, reporting moderate results. Some scholars have proposed that a combined clinical-actuarial approach would be optimal for ED settings, allowing clinicians to use the empirical categories set out in an actuarial tool to aid, rather than replace, clinical judgment.
      • Luck L.
      • Jackson D.
      • Usher K.
      STAMP: components of observable behaviour that indicate potential for patient violence in emergency departments.
      In contrast, emergency nurses have expressed the need for a standardized tool that focuses on objective risk factors, particularly as ED risk assessments must be rapid.
      • Cabilan C.J.
      • Johnston A.N.
      Review article: identifying occupational violence patient risk factors and risk assessment tools in the emergency department: a scoping review.
      Other studies have similarly concluded that clinicians prefer risk assessment to contain an element of structure, with some suggesting that reliance on clinical judgment alone puts less experienced staff at a disadvantage.
      • Woods P.
      Risk assessment and management approaches on mental health units.
      In fact, numerous studies have found that staff with less experience (both clinically and in the emergency department specifically) are more likely to experience patient violence in emergency settings.
      • Niu S.F.
      • Kuo S.F.
      • Tsai H.T.
      • Kao C.C.
      • Traynor V.
      • Chou K.R.
      Prevalence of workplace violent episodes experienced by nurses in acute psychiatric settings.
      ,
      • ALBashtawy M.
      Workplace violence against nurses in emergency departments in Jordan.
      Cabilan et al
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      point out that a structured approach to risk assessment does not preclude sensitivity to context and argue that a multidimensional approach, addressing both static and dynamic risk factors, is most appropriate.
      Even if a tool improves violence prediction, if it is not implemented properly, it is essentially useless. We found variability in levels of implementation but cannot identify why this was the case. Usability of the BARS and the EDART were examined, with both reporting positive findings,
      • Campbell E.
      • Jessee D.
      • Whitney J.
      • Vupputuri S.
      • Carpenter J.
      Development and implementation of an emergent documentation aggression rating tool: quality improvement.
      ,
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      whereas an evaluation of nurses’ confidence and perceived ability to prevent violence before and after implementation of the BOC reported mixed findings.
      • Senz A.
      • Ilarda E.
      • Klim S.
      • Kelly A.M.
      Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
      None of the included studies explicitly assessed feasibility or acceptability. Whereas lengthy risk assessment tools may be impractical in ED settings,
      • Cabilan C.J.
      • McRae J.
      • Learmont B.
      • et al.
      Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
      the BARS, a single-item tool, had low adoption.
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      ,
      • Legambi T.F.
      • Doede M.
      • Michael K.
      • Zaleski M.
      A quality improvement project on agitation management in the emergency department.
      Lack of understanding and enthusiasm for the tool were cited as possible reasons for this outcome, perhaps pointing to the importance of a strong implementation strategy.
      • Schumacher J.A.
      • Gleason S.H.
      • Holloman G.H.
      • McLeod W.T.
      Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
      The true success of these tools should, of course, ultimately be measured in terms of reductions in violence rather than simply its prediction. Patient violence is harmful in and of itself, yet the interventions used to manage patient violence can be equally damaging. The use of physical, mechanical, and chemical restraint can be physically and psychologically harmful to all involved.
      • Sethi F.
      • Parkes J.
      • Baskind E.
      • Paterson B.
      • O’Brien A.
      Restraint in mental health settings: is it time to declare a position?.
      This review found no or nonsignificant reductions in violence after tool implementation, but this is based on limited and poor-quality evidence, so no firm conclusions can be drawn. Measuring outcomes in terms of restraint use or emergency security responses is, in our view, mistaken because the aim of prediction is to facilitate the early intervention of less coercive measures.
      The only strong recommendation that we can make as a result of this review is about what needs to be done to address our identified gap in the literature. Ideally, large-scale, multisite randomized controlled trials are needed to provide good-quality evidence on the use of violence risk assessment tools in emergency settings, exploring their efficacy in terms of predicting and also reducing violent incidents. Based on the recency of the included literature, we anticipate that small-scale studies will continue to proliferate, and we hope that in the not-too-distant future, systematic review with meta-analysis will be achievable.

      Strengths and Limitations

      The strength of our findings is limited by the quality of the included studies. However, the lack of strong evidence in this area is a significant finding in itself. By excluding unpublished literature, we may have missed relevant research, although we sought to mitigate this by directly contacting the authors of all relevant unpublished literature to ascertain whether the work was taken further. Finally, the generalizability of our results is limited by the geographical distribution of our included studies, which were all conducted in the United States or Australia. Given the significant body of literature exploring patient violence globally,
      • Spector P.E.
      • Zhou Z.E.
      • Che X.X.
      Nurse exposure to physical and nonphysical violence, bullying, and sexual harassment: a quantitative review.
      it was disappointing that we could not capture any evidence about violence risk assessment more widely. Similarly, the fact that no studies took place in the AMU limits the assumptions we can make about the tools’ suitability for this clinical area. By uncovering these gaps in the literature, this review has highlighted important areas for future research.

      Implications for Emergency Nursing

      Violence risk assessment can identify patients in emergency care settings who are at risk of becoming violent. However, there is currently insufficient high-quality evidence to draw conclusions about the predictive capacity, acceptability, feasibility, and usability of existing tools in emergency care settings. In the meantime, researchers and emergency nurses looking to implement violence risk assessment strategies should take steps to ensure a strong implementation strategy to maximize uptake. Such strategies may include the use of a violence risk assessment tool, and, in the absence of any strong evidence for choosing one over another, we recommend choosing the tool that aligns most strongly with the specific context it will be used in.

      Conclusion

      Patient-perpetrated violence is a significant problem in emergency care settings globally. Despite its prevalence, there is a paucity of high-quality evidence evaluating the psychometric properties of violence risk assessment tools currently used along the emergency care pathway. Multiple tools exist, however, and the recency of much of the evidence evaluating their effectiveness indicates that this clinical issue is gaining traction. There is a long way to go before violence risk assessment is as established in emergency care settings as it is in mental health settings. Finding out which tools are most effective in predicting and preventing violence would be a good starting point; the evidence to support choosing one tool over another is not yet available, but the evidence from this review suggests that we are well on our way.

      Data, Code, and Research Materials Availability

      We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing, we confirm that we have followed the regulations of our institutions concerning intellectual property.

      Author Disclosures

      Conflicts of interest: None to report.
      The study from which this review came, the “Violence in acute medical units and emergency departments (VoicED)” study, was funded by the Clive Richards Foundation, previously the Clive and Sylvia Richards Charity, Hereford, UK [grant number CSRC200135]. The funding comprised salary for a research associate as well as transcription and dissemination costs. Funding was not related to any specific research activity.

      Appendix.

      Google Scholar

      NB. 256 character limit
      2007-2021: ((risk AND assess) OR (risk AND tool) OR (risk AND instrument)) AND (emergency OR “acute medical unit”) AND (violence OR aggression OR assault OR attack OR abuse) AND (psychometric OR validity OR reliability OR predictability OR feasibility OR usability)
      Results then limited to top 200 (by relevance)
      Supplementary Table 1
      DATABASE: MEDLINE
      Search terms:
      1Subject headingsRisk assessment
      Keywordsrisk∗ adj3 assess∗, risk∗ adj3 screen∗, risk∗ adj3 checklist∗, risk∗ adj3 tool∗, risk∗ adj3 scale∗, risk∗ adj3 measur∗, risk∗ adj3 instrument∗, "Staring and eye contact, Tone and volume of voice, Anxiety, Mumbling, and Pacing", STAMP, "17-cue assessment tool", "17-cue violence assessment tool", "Staring, Tone, Anxiety, Mumbling, Pacing, Emotions, Disease progress, Assertive, Resources", STAMPEDAR, "Violence Risk Screen Decision Support in triage", VRSDSiT, "Broset Violence Checklist", BVC



      AND
      2Subject headingsEmergency Medical Services, Emergency Service Hospital [exp]
      Keywords“emergency room∗”, "emergency department∗", "emergency service∗", "emergency ward∗", "emergency care", "accident and emergency", "accident & emergency", "emergency health service∗", triag∗, “ED”, “ER”, “A&E”, “acute medical unit∗”, “AMU”, “clinical decision unit∗”, “CDU”, “acute admissions unit∗”, “acute assessment unit∗”, “AAU”, “acute medical receiving unit∗”, “AMRU”, “assessment and diagnostic unit∗”, “ADU”, “emergency assessment unit∗”, “EAU”, “emergency care unit∗”, “ECU”, “EMAU”, “medical assessment unit∗”, “MAU”, “medical assessment and planning unit∗”, “MAPU”, “medical admissions unit∗”



      AND
      3Subject headingsWorkplace violence, Aggression [exp], Violence
      Keywordsviolen∗, aggress∗, assault∗, attack∗, harass∗, verbal adj3 abus∗, physical adj3 abus∗, "verbal hostility"



      AND
      4Subject headingsPsychometrics, Reproducibility of results [exp]
      Keywords“psychometric properties”, valid∗, reliab∗, "internal∗ consisten∗", feasib∗, acceptab∗, usab∗, predict∗, evaluat∗
      Key: Commas indicate terms combined with OR; [exp] = search term exploded
      Supplementary Table 2
      DATABASE: Embase
      Search terms:
      1Subject headingsRisk assessment [exp]
      Keywordsrisk∗ adj3 assess∗, risk∗ adj3 screen∗, risk∗ adj3 checklist∗, risk∗ adj3 tool∗, risk∗ adj3 scale∗, risk∗ adj3 measur∗, risk∗ adj3 instrument∗, "Staring and eye contact, Tone and volume of voice, Anxiety, Mumbling, and Pacing", STAMP, "17-cue assessment tool", "17-cue violence assessment tool", "Staring, Tone, Anxiety, Mumbling, Pacing, Emotions, Disease progress, Assertive, Resources", STAMPEDAR, "Violence Risk Screen Decision Support in triage", VRSDSiT, "Broset Violence Checklist", BVC



      AND
      2Subject headingsEmergency Health Service [exp], Emergency Ward [exp]
      Keywords“emergency room∗”, "emergency department∗", "emergency service∗", "emergency ward∗", "emergency care", "accident and emergency", "accident & emergency", "emergency health service∗", "triag∗", “ED”, “ER”, “A&E”, “acute medical unit∗”, “AMU”, “clinical decision unit∗”, “CDU”, “acute admissions unit∗”, “acute assessment unit∗”, “AAU”, “acute medical receiving unit∗”, “AMRU”, “assessment and diagnostic unit∗”, “ADU”, “emergency assessment unit∗”, “EAU”, “emergency care unit∗”, “ECU”, “EMAU”, “medical assessment unit∗”, “MAU”, “medical assessment and planning unit∗”, “MAPU”, “medical admissions unit∗”



      AND
      3Subject headingsWorkplace violence {prevention}, Aggression {prevention}, Violence {prevention}, Verbal hostility {prevention}, Assault {prevention}
      Keywordsviolen∗, aggress∗, assault∗, attack∗, harass∗, verbal∗ adj3 abus∗, physical∗ adj3 abus∗, "verbal hostility”



      AND
      4Subject headingsPsychometry [exp], Reproducibility [exp], Validity [exp], Reliability [exp], Usability
      Keywords“psychometric properties”, valid∗, reliab∗, "internal∗ consisten∗", feasib∗, acceptab∗, usab∗, predict∗, evaluat∗
      Key: Commas indicate terms combined with OR; [exp] = search term exploded; {text in braces} = subheadings selected (NB. where not specified, all subheadings were included)
      Supplementary Table 3
      DATABASE: Web of Science
      Search terms:
      1AND (TS=(risk∗ NEAR/3 assess∗) OR TS=(risk∗ NEAR/3 screen∗) OR TS=(risk∗ NEAR/3 checklist∗) OR TS=(risk∗ NEAR/3 tool∗) OR TS=(risk∗ NEAR/3 scale∗) OR TS=(risk∗ NEAR/3 measur∗) OR TS=(risk∗ NEAR/3 instrument∗) OR TS=("Staring and eye contact, Tone and volume of voice, Anxiety, Mumbling, and Pacing") OR TS=(STAMP) OR TS=("17-cue assessment tool") OR TS=("17-cue violence assessment tool") OR TS=("Staring, Tone, Anxiety, Mumbling, Pacing, Emotions, Disease progress, Assertive, Resources") OR TS=(STAMPEDAR) OR TS=("Violence Risk Screen Decision Support in triage") OR TS=(VRSDSiT) OR TS=("Broset Violence Checklist") OR TS=(BVC))
      2AND (TS=(“emergency room∗”) OR TS=("emergency department∗") OR TS=("emergency service∗") OR TS=("emergency ward∗") OR TS=("emergency care") OR TS=("accident and emergency") OR TS=("accident & emergency") OR TS=("emergency health service∗") OR TS=("ED") OR TS=("ER") OR TS=("A&E") OR TS=(“acute medical unit∗”) OR TS=("AMU") OR TS=(“clinical decision unit∗”) OR TS=("CDU") OR TS=(“acute admissions unit∗”) OR TS=(“acute assessment unit∗”) OR TS=("AAU") OR TS=(“acute medical receiving unit∗”) OR TS=("AMRU") OR TS=(“assessment and diagnostic unit∗”) OR TS=("ADU") OR TS=(“emergency assessment unit∗”) OR TS=("EAU") OR TS=(“emergency care unit∗”) OR TS=("ECU") OR TS=("EMAU") OR TS=(“medical assessment unit∗”) OR TS=("MAU") OR TS=(“medical assessment and planning unit∗”) OR TS=("MAPU") OR TS=(“medical admissions unit∗”))
      3AND
      4(TS=("psychometric properties") OR TS=(valid∗) OR TS=(reliab∗) OR TS=("internal∗ consisten∗") OR TS=(feasib∗) OR TS=(acceptab∗) OR TS=(usab∗) OR TS=(predict∗) OR TS=(evaluat∗))
      Key: TS = Searched in ‘Topic’ field
      Supplementary Table 4
      DATABASE: CINAHL Plus
      Search terms:
      1Subject headingsRisk assessment, Clinical assessment tools
      Keywordsrisk∗ adj3 assess∗, risk∗ adj3 screen∗, risk∗ adj3 checklist∗, risk∗ adj3 tool∗, risk∗ adj3 scale∗, risk∗ adj3 measur∗, risk∗ adj3 instrument∗, "Staring and eye contact, Tone and volume of voice, Anxiety, Mumbling, and Pacing", STAMP, "17-cue assessment tool", "17-cue violence assessment tool", "Staring, Tone, Anxiety, Mumbling, Pacing, Emotions, Disease progress, Assertive, Resources", STAMPEDAR, "Violence Risk Screen Decision Support in triage", VRSDSiT, "Broset Violence Checklist", BVC



      AND
      2Subject headingsEmergency Service, Emergency Medical Services
      Keywords“emergency room∗”, "emergency department∗", "emergency service∗", "emergency ward∗", "emergency care", "accident and emergency", "accident & emergency", "emergency health service∗", "triag∗", “ED”, “ER”, “A&E”, “acute medical unit∗”, “AMU”, “clinical decision unit∗”, “CDU”, “acute admissions unit∗”, “acute assessment unit∗”, “AAU”, “acute medical receiving unit∗”, “AMRU”, “assessment and diagnostic unit∗”, “ADU”, “emergency assessment unit∗”, “EAU”, “emergency care unit∗”, “ECU”, “EMAU”, “medical assessment unit∗”, “MAU”, “medical assessment and planning unit∗”, “MAPU”, “medical admissions unit∗”



      AND
      3Subject headingsWorkplace violence, Aggression, Violence, Verbal abuse, Patient assault, Assault and battery
      Keywordsviolen∗, aggress∗, assault∗, attack∗, harass∗, verbal adj3 abus∗, physical adj3 abus∗, "verbal hostility"



      AND
      4Subject headingsPsychometrics, Measurement issues and assessments [exp]
      Keywords“psychometric properties”, valid∗, reliab∗, "internal∗ consisten∗", feasib∗, acceptab∗, usab∗, predict∗, evaluat∗
      Key: Commas indicate terms combined with OR; [exp] = search term exploded

      Supplementary Data

      References

      1. Violence and aggression in the NHS: estimating the size and impact of the problem- an interim report. Royal College of Nursing, 2018 (Published October 31, 2018. Accessed April 20, 2021.)
      2. Employment survey. Royal College of Nursing, 2019 (Published)
        • Ashton R.A.
        • Morris L.
        • Smith I.
        A qualitative meta-synthesis of emergency department staff experiences of violence and aggression.
        Int Emerg Nurs. 2018; 39: 13-19https://doi.org/10.1016/j.ienj.2017.12.004
        • Pich J.V.
        • Kable A.
        • Hazelton M.
        Antecedents and precipitants of patient-related violence in the emergency department: results from the Australian VENT Study (Violence in Emergency Nursing and Triage).
        Australas Emerg Nurs J. 2017; 20: 107-113https://doi.org/10.1016/j.aenj.2017.05.005
      3. Reducing violence and aggression in A&E through a better experience. Design Council, CABE, Department of Health, 2011 (Published)
      4. Violence against frontline NHS staff: Research Study Conducted for COI on behalf of the NHS Security Management Service. Ipsos MORI, 2010 (Published June 24, 2010)
        • Hassankhani H.
        • Parizad N.
        • Gacki-Smith J.
        • Rahmani A.
        • Mohammadi E.
        The consequences of violence against nurses working in the emergency department: a qualitative study.
        Int Emerg Nurs. 2018; 39: 20-25https://doi.org/10.1016/j.ienj.2017.07.007
        • Monks R.
        • Topping A.
        • Newell R.
        The dissonant care management of illicit drug users in medical wards, the views of nurses and patients: a grounded theory study.
        J Adv Nurs. 2013; 69: 935-946https://doi.org/10.1111/j.1365-2648.2012.06088.x
        • Roche M.
        • Diers D.
        • Duffield C.
        • Catling-Paull C.
        Violence toward nurses, the work environment, and patient outcomes.
        J Nurs Scholarsh. 2010; 42: 13-22https://doi.org/10.1111/j.1547-5069.2009.01321.x
        • D’Ettorre G.
        • Pellicani V.
        • Mazzotta M.
        • Vullo A.
        Preventing and managing workplace violence against healthcare workers in emergency departments.
        Acta Biomed. 2018; 89: 28-36https://doi.org/10.23750/abm.v89i4-s.7113
        • Sharifi S.
        • Shahoei R.
        • Nouri B.
        • Almvik R.
        • Valiee S.
        Effect of an education program, risk assessment checklist and prevention protocol on violence against emergency department nurses: a single center before and after study.
        Int Emerg Nurs. 2020; 50: 100813https://doi.org/10.1016/j.ienj.2019.100813
        • Kim S.C.
        • Ideker K.
        • Todicheeney-Mannes D.
        Usefulness of Aggressive Behaviour Risk Assessment Tool for prospectively identifying violent patients in medical and surgical units.
        J Adv Nurs. 2012; 68: 349-357https://doi.org/10.1111/j.1365-2648.2011.05744.x
        • Cabilan C.J.
        • Johnston A.N.
        Review article: identifying occupational violence patient risk factors and risk assessment tools in the emergency department: a scoping review.
        Emerg Med Australas. 2019; 31: 730-740https://doi.org/10.1111/1742-6723.13362
        • Woods P.
        • Almvik R.
        The Brøset Violence Checklist (BVC).
        Acta Psychiatr Scand Suppl. 2002; 106: 103-105https://doi.org/10.1034/j.1600-0447.106.s412.22.x
        • Dickens G.L.
        • O’Shea L.E.
        • Christensen M.
        Structured assessments for imminent aggression in mental health and correctional settings: systematic review and meta-analysis: risk assessment for imminent violence.
        Int J Nurs Stud. 2020; 104: 103526https://doi.org/10.1016/j.ijnurstu.2020.103526
        • Partridge B.
        • Affleck J.
        Predicting aggressive patient behaviour in a hospital emergency department: an empirical study of security officers using the Brøset Violence Checklist.
        Australas Emerg Care. 2018; 21: 31-35https://doi.org/10.1016/j.auec.2017.11.001
        • Ginsburg A.S.
        • Tawiah Agyemang C.
        • Ambler G.
        • et al.
        mPneumonia, an innovation for diagnosing and treating childhood pneumonia in low-resource settings: a feasibility, usability and acceptability study in Ghana.
        PLoS One. 2016; 11e0165201https://doi.org/10.1371/journal.pone.0165201
        • Page M.J.
        • McKenzie J.E.
        • Bossuyt P.M.
        • et al.
        The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
        BMJ. 2021; 372: n71https://doi.org/10.1136/bmj.n71
        • Bramer W.M.
        • Rethlefsen M.L.
        • Kleijnen J.
        • Franco O.H.
        Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study.
        Syst Rev. 2017; 6: 245https://doi.org/10.1186/s13643-017-0644-y
        • Sterne J.A.C.
        • Hernán M.A.
        • Reeves B.C.
        • et al.
        ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions.
        BMJ. 2016; 355: i4919https://doi.org/10.1136/bmj.i4919
        • Boateng G.O.
        • Neilands T.B.
        • Frongillo E.A.
        • Melgar-Quiñonez H.R.
        • Young S.L.
        Best practices for developing and validating scales for health, social, and behavioral research: a primer.
        Front Public Health. 2018; 6: 149https://doi.org/10.3389/fpubh.2018.00149
        • McHugh M.L.
        Interrater reliability: the kappa statistic.
        Biochem Med (Zagreb). 2012; 22: 276-282https://doi.org/10.11613/BM.2012.031
        • Schumacher J.A.
        • Gleason S.H.
        • Holloman G.H.
        • McLeod W.T.
        Using a single-item rating scale as a psychiatric behavioral management triage tool in the emergency department.
        J Emerg Nurs. 2010; 36: 434-438https://doi.org/10.1016/j.jen.2010.01.013
        • Campbell E.
        • Jessee D.
        • Whitney J.
        • Vupputuri S.
        • Carpenter J.
        Development and implementation of an emergent documentation aggression rating tool: quality improvement.
        J Emerg Nurs. 2021; 47: 696-706https://doi.org/10.1016/j.jen.2021.04.011
        • Legambi T.F.
        • Doede M.
        • Michael K.
        • Zaleski M.
        A quality improvement project on agitation management in the emergency department.
        J Emerg Nurs. 2021; 47: 390-399.e393https://doi.org/10.1016/j.jen.2021.01.005
        • Senz A.
        • Ilarda E.
        • Klim S.
        • Kelly A.M.
        Development, implementation and evaluation of a process to recognise and reduce aggression and violence in an Australian emergency department.
        Emerg Med Australas. 2021; 33: 665-671https://doi.org/10.1111/1742-6723.13702
        • Cabilan C.J.
        • McRae J.
        • Learmont B.
        • et al.
        Validity and reliability of the novel three-item occupational violence patient risk assessment tool.
        J Adv Nurs. 2022; 78: 1176-1185https://doi.org/10.1111/jan.15166
        • Connor M.
        • Armbruster M.
        • Hurley K.
        • Lee E.
        • Chen B.
        • Doering L.
        Diagnostic sensitivity of the dynamic appraisal of situational aggression to predict violence and aggression by behavioral health patients in the emergency department.
        J Emerg Nurs. 2020; 46: 302-309https://doi.org/10.1016/j.jen.2019.12.006
        • Jackson D.
        • Wilkes L.
        • Luck L.
        Cues that predict violence in the hospital setting: findings from an observational study.
        Collegian. 2014; 21: 65-70https://doi.org/10.1016/j.colegn.2013.02.006
        • Swift R.H.
        • Harrigan E.P.
        • Cappelleri J.C.
        • Kramer D.
        • Chandler L.P.
        Validation of the Behavioural Activity Rating Scale (BARS): a novel measure of activity in agitated patients.
        J Psychiatr Res. 2002; 36: 87-95https://doi.org/10.1016/s0022-3956(01)00052-8
        • Ogloff J.R.
        • Daffern M.
        The dynamic appraisal of situational aggression: an instrument to assess risk for imminent aggression in psychiatric inpatients.
        Behav Sci Law. 2006; 24: 799-813https://doi.org/10.1002/bsl.741
        • Spector P.E.
        • Zhou Z.E.
        • Che X.X.
        Nurse exposure to physical and nonphysical violence, bullying, and sexual harassment: a quantitative review.
        Int J Nurs Stud. 2014; 51: 72-84https://doi.org/10.1016/j.ijnurstu.2013.01.010
        • Griffith J.J.
        • Daffern M.
        • Godber T.
        Examination of the predictive validity of the dynamic appraisal of situational aggression in two mental health units.
        Int J Ment Health Nurs. 2013; 22: 485-492https://doi.org/10.1111/inm.12011
        • Dumais A.
        • Larue C.
        • Michaud C.
        • Goulet M.H.
        Predictive validity and psychiatric nursing Staff’s perception of the clinical usefulness of the French version of the dynamic appraisal of situational aggression.
        Issues Ment Health Nurs. 2012; 33: 670-675https://doi.org/10.3109/01612840.2012.697254
        • Lantta T.
        • Kontio R.
        • Daffern M.
        • Adams C.
        • Välimäki M.
        Using the dynamic appraisal of situational aggression with mental health inpatients: a feasibility study.
        Patient Prefer Adherence. 2016; 10: 691-701https://doi.org/10.2147/ppa.s103840
        • Trenoweth S.
        Perceiving risk in dangerous situations: risks of violence among mental health inpatients.
        J Adv Nurs. 2003; 42: 278-287https://doi.org/10.1046/j.1365-2648.2003.02617.x
        • Llor-Esteban B.
        • Sánchez-Muñoz M.
        • Ruiz-Hernández J.A.
        • Jiménez-Barbero J.A.
        User violence towards nursing professionals in mental health services and emergency units.
        Eur J Psychol Appl Leg Context. 2017; 9: 33-40https://doi.org/10.1016/j.ejpal.2016.06.002
        • Webster C.D.
        • Martin M.
        • Brink J.
        • Nicholls T.L.
        • Desmarais S.L.
        Manual for the Short Term Assessment of Risk and Treatability (START).
        British Columbia Mental Health and Addiction Services, Coquitlam, Canada2009
        • Doyle M.
        • Dolan M.
        Violence risk assessment: combining actuarial and clinical information to structure clinical judgements for the formulation and management of risk.
        J Psychiatr Ment Health Nurs. 2002; 9: 649-657https://doi.org/10.1046/j.1365-2850.2002.00535.x
        • Luck L.
        • Jackson D.
        • Usher K.
        STAMP: components of observable behaviour that indicate potential for patient violence in emergency departments.
        J Adv Nurs. 2007; 59: 11-19https://doi.org/10.1111/j.1365-2648.2007.04308.x
        • Woods P.
        Risk assessment and management approaches on mental health units.
        J Psychiatr Ment Health Nurs. 2013; 20: 807-813https://doi.org/10.1111/jpm.12022
        • Niu S.F.
        • Kuo S.F.
        • Tsai H.T.
        • Kao C.C.
        • Traynor V.
        • Chou K.R.
        Prevalence of workplace violent episodes experienced by nurses in acute psychiatric settings.
        PLoS One. 2019; 14e0211183https://doi.org/10.1371/journal.pone.0211183
        • ALBashtawy M.
        Workplace violence against nurses in emergency departments in Jordan.
        Int Nurs Rev. 2013; 60: 550-555https://doi.org/10.1111/inr.12059
        • Sethi F.
        • Parkes J.
        • Baskind E.
        • Paterson B.
        • O’Brien A.
        Restraint in mental health settings: is it time to declare a position?.
        Br J Psychiatry. 2018; 212: 137-141https://doi.org/10.1192/bjp.2017.31

      Biography

      Dana Sammut is a Research Associate, Institute of Clinical Sciences, University of Birmingham, Birmingham, West Midlands. Twitter: @DanaSammut. ORCID identifier: https://orcid.org/0000-0002-6593-4782.
      Nutmeg Hallett is a Lecturer in Nursing, Institute of Clinical Sciences, University of Birmingham, Birmingham, West Midlands. Twitter: @dr_nutmeg. ORCID identifier: https://orcid.org/0000-0003-3115-8831.
      Liz Lees-Deutsch is a Associate Professor for Nursing and Clinical Academic Nurse, Center for Care Excellence, University Hospitals Coventry and Warwickshire, Coventry, West Midlands. Twitter: @LizzieDeutsch. ORCID identifier: https://orcid.org/0000-0002-7246-0503.
      Geoff Dickens is a Professor, Mental Health Nursing, Department of Nursing, Midwifery and Health, Northumbria University, Newcastle-Upon-Tyne, Tyne & Wear. ORCID identifier: https://orcid.org/0000-0002-8862-1527.