Ex) Article Title, Author, Keywords
Ex) Article Title, Author, Keywords
R Clin Pharm 2023; 1(1): 40-48
Published online June 30, 2023 https://doi.org/10.59931/rcp.23.005
Copyright © Asian Conference On Clinical Pharmacy.
Ahyoung Lee , Ji Min Han , Kwanghee Jun , Kyu Nam Heo , McKenzie S. Grinalds , Natalie C. Washburn , Todd A. Walroth , Young-Mi Ah , Ju-Yeun Lee
Correspondence to:Ju-Yeun Lee
E-mail jypharm@snu.ac.kr
ORCID
https://orcid.org/0000-0002-2261-7330
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/bync/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background: There is no objective and standardized tool for stratification of high-alert medications (HAMs) that reflect each institution’s practice of medication management, drug utilization in the institution, and patient safety protocols in the Korean hospital settings. We aimed to develop the Korean version of the high-alert medication stratification tool (K-HAMST) and assess its content validity and reliability.
Methods: Ten clinical pharmacists from multi-site hospitals completed a two-round Delphi survey to assess the content validity of the translated High-Alert Medication Stratification Tool-Revised. Content validity was demonstrated using the item content validity index and scale content validity index (S-CVI). An expert meeting was conducted to revise the tool to accommodate the clinical practice and workflow in Korea based on the results of the content validity index. Reliability was assessed by calculating the risk scores for 37 HAMs and 37 control medications. The interrater reliability of each medication was assessed using the Kendall’s coefficient of concordance (W).
Results: The initial S-CVI was 0.71. After revision, the final S-CVI of the K-HAMST was 0.92, indicating that the tool has content validity. The HAM scores ranged from 3 to 8 (n=37; median [interquartile {IQR}, 4 [3–5]), whereas the control medication scores ranged from 1 to 2 (n=37; median [IQR], 1 [1–1]). The Kendall’s coefficient of concordance (W) was 0.57, indicating moderate agreement between raters (p<0.001).
Conclusion: The K-HAMST is a valid and reliable tool for assessing and evaluating HAMs in hospital settings.
KeywordsContent validity; HAM identification; High-alert medication; Medication safety; Risk stratification tool
The Institute for Safe Medication Practices (ISMP) defines “high-alert medications (HAMs) as drugs that bear a heightened risk of causing significant patient harm when used in error” [1]. According to a review of events in a database of 317 preventable adverse drug events (ADEs), approximately 60% of the preventable ADEs resulted from the use of HAMs [2]. In a trigger-based ADE identification study conducted in Spain, 74.5% of all ADEs identified in geriatric patients with multimorbidity were attributed to HAMs [3]. As previous studies have shown [4,5], the consequences of errors with HAMs are more devastating to patients since they contribute to a substantial number of preventable ADEs compared to other medications.
HAMs are directly associated with patient safety risks in the healthcare system, which is a strong trigger for each country to develop HAM lists and management guidelines. Previously, researchers reported HAM lists and tools for the identification of such medications [6-9]. In 2015, to support the development and modification of an institutional HAM list, the American Society of Health-System Pharmacists (ASHP) Inpatient Care Practitioner’s Medication Safety Advisory Group distributed an objective high-alert medication checklist. The purpose of the ASHP checklist is to create or modify each organization’s HAM list to meet the needs of a specific institution [10].
Eskenazi Health in the United States developed a standardized high-alert medication stratification tool (HAMST-R) based on the aforementioned ASHP checklist after subsequent validation and interrater reliability testing. According to the HAMST-R study, the HASMT-R consists of 10 items; items 1–4 evaluate overall patient safety issues while items 5–10 evaluate safety concerns in the steps of the drug use process. If more than 50% of the questions within each step of the drug use process are answered “yes”, 1 point is added for that specific phase. The total score adds up to 10 when all phases receive a point, and a HAMST-R score of 4 or higher identifies a HAM [8]. The HAMST-R, developed in the United States, was shown to be a valid and reliable tool for evaluating medication risks in hospital settings across the United States and Canada [8,9].
In Korea, there is no objective and standardized tool for stratification of HAMs in hospital settings. The Korea Institute for Healthcare Accreditation (KOIHA) suggests HAM categories including sedative drugs, anticancer drugs, high-concentration electrolytes, antithrombotic agents for injection, insulin, and contrast agents based on expert opinion [11]. These Korean-specific HAM categories are insufficient to reflect each institution’s practice of medication management, drug utilization in the institution, and patient safety protocols. Among the numerous tools to identify HAMs, the HAMST-R is a structured and standardized risk stratification tool. We adjusted HAMST-R, known as HAMST-R PRO, to be used prospectively and published the results [12]. Furthermore, the tool can be used for analyses of safety practices of HAMs by evaluating the medication use process of prescribing, transcribing, dispensing, and administration as outlined in questions 5 to 10 of HAMST-R [8,9]. A heightened emphasis on the management of HAMs and insufficient HAMs identification tools in Korea promotes the development of an objective and quantitative tool that can be utilized across the healthcare systems in Korea.
In this study, we developed the K-HAMST based on Washburn et al. [8] study of the HAMST-R and tested the validity and interrater reliability of K-HAMST by referring to the phase 2 study of the HAMST-R by Shenk et al. [9]. The following steps were conducted: 1) translation and back-translation, 2) content validity testing using the Delphi method, and 3) interrater reliability testing using the K-HAMST.
An advisory group consisting of ten clinical pharmacists from medical institutions was recommended by the Patient Safety Quality Improvement Committee under the Korean Society of Health-System Pharmacists (KSHP). All the recommended pharmacists had more than 5 years of experience in hospital pharmacy. Of the ten pharmacists, interrater reliability verification was conducted by six clinical pharmacists from six independent tertiary hospitals in Korea.
In this study, the translation process was established using translation, back-translation, and proofreading performed by the researchers. Back-translation is a well-known method for maintaining equivalence between original and translated versions of tools and questionnaires [13]. The HAMST-R was translated into Korean by two bilingual researchers conducting this study. Back-translation was performed by translating the Korean version back into English without referring to the original tool. During the proofreading process, the back-translated English version of the tool was compared to HAMST-R to validate whether all meanings of the subitems were translated equivalently.
Content validity refers to the method of estimating the validity of a new or revised tool quantitatively. A widely used method is known as the content validity index, which is based on expert rating for each item of the scale. An item content validity index (I-CVI) represents the validity of an item on a scale, whereas the scale content validity index (S-CVI) represents the validity of the overall scale [14]. Ten clinical pharmacists completed a two-round Delphi survey to assess the content validity of the K-HAMST translated and adapted from the HAMST-R. The study participants provided consent to enroll in the two-round Delphi process. An initial online Delphi survey to evaluate content validity was conducted on the 30 sub-items of the K-HAMST. The online survey consisted of 30 questions to assess the validity of the tool with an ordinal scale from 1 to 4, allowing the participants to provide opinions on each item. The 4-point ordinal scale denotes 1 (not relevant), 2 (somewhat relevant), 3 (quite relevant), and 4 (highly relevant). If an I-CVI was higher than 79 percent, the item was appropriate, and if it was between 70 and 79 percent, the item required revision [15]. The tool was considered to have content validity if the overall S-CVI was higher than 0.80 [16].
If the I-CVI was less than 70 percent, we revised the question instead of being eliminated. After the first online survey, an expert meeting was conducted involving ten study participants to revise the tool to accommodate the medical situation and drug management protocols in Korea by focusing on questions with an I-CVI of less than 0.79. For the items that received a low I-CVI of less than 0.79, the group went through an extensive discussion to modify items or add examples to each item. The modified items specific to medical practices in Korea may guide the evaluators to easily assess and confirm whether the medication in interest is considered HAM or not. The second Delphi survey to test the final validation of the revised questions using content validity rating was conducted with a similar procedure as the first survey.
The consistency or agreement among the experts interpreting the data is important. The extent of agreement among data collectors is called, “
We used a sample size of 37 for each group, namely the HAMs and controls, to obtain 80% power as done in the previous study of the HAMST-R [8]. A comprehensive list of HAMs and control medications (non-HAMs) was derived from composite list of HAMs from the ISMP, Australian Commission on Safety and Quality in Health Care (APINCHS), and the HAMST-R study [1,7,8]. We have excluded medications that are not marketed or routinely used in Korea from this composite list and we finalized the list with the expert opinion of 10 clinical pharmacists from six different hospitals, considering both the frequency of use in hospitals and the extent of harm to the patient. Our list was used to determine the interrater reliability of the K-HAMST and to determine a cut-off score specific for use in hospitals in Korea. The interrater reliability of the K-HAMST was studied by six clinical pharmacists selected to represent six different institutions. Reliability was assessed by calculating the risk scores for 37 HAMs and 37 control medications. Prior to the utilization of the tool, the researchers provided the participants with instructions on how to answer each question and provided a list of HAMs and control medications, including specific formulations and product brand names. The study participants evaluated 74 medications and rated each medication based on national and local hospital guidelines for the management of medications. The K-HAMST questions were scored on four general patient safety concerns and six phases of the medication use process, similar to the previously developed HAMST-R. Similar to the previous study, each phase consisted of a different number of questions. Directions stated that if more than 50% of the questions within each phase of the medication use process were answered “yes”, 1 point was given for the specific phase. The maximum total score was 10 when all phases received a score [8,17].
The Anderson–Darling test was performed to test the normality of the data. The significance of variance between data was determined using the Mann-Whitney U test with an alpha value of less than 0.05. In this study, I-CVI was computed as the number of experts giving a rating 3 or 4 to the relevancy of each item, divided by the total number of experts. The S-CVI was the average of the I-CVI score [16]. The final score was the median of the scores evaluated for each site, and the median score was rounded up to the nearest whole number. The threshold score for the I-CVI was less than 0.79, indicating that the content of the item was not relevant and needed modification. To measure how consistently the raters evaluated each medication, the interrater reliability of the risk scores of each medication was studied using the Kendall’s coefficient of concordance (W) method. The value of W ≤0 indicates no agreement, 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement [17]. The cutoff score to determine HAMs was established based on the risk scores of the HAMs and control medications (N=74) evaluated by six hospital pharmacists. The threshold was based on a score that included only HAMs and no control medications. Statistical analysis was performed using the SPSS version 22.0 software (IBM Corporation, Armonk, NY, USA).
The overview of the final version of K-HAMST in English is shown in Supplementary Table 1. The results of the translation of 30 questions are shown in Supplementary Table 2. Based on the results of the first-round Delphi survey, modification of the questions and addition of references or examples was done to add clarification to each question. The results of the modified K-HAMST that reflect expert opinion are shown in Supplementary Table 3.
During the first Delphi survey, the I-CVI scores ranged from 0.4–0.9, and 14 of 30 items that received an I-CVI greater than 0.79 met the criteria for significance (Fig. 1). The overall validity of the tool after the first round of the Delphi survey was 0.71, indicating that the S-CVI of the scale before revision did not have content validity since it was less than 0.80 [16]. Of the total of 30 questions, 16 questions that received I-CVI less than 0.79 went through the modification process. After the revision of the tool, the content validity of the K-HAMST was re-evaluated using the second Delphi survey. All 30 items achieved an I-CVI greater than 0.79, with scores that ranged from 0.8–1.0. The S-CVI score increased to 0.92, indicating that the tool had content validity. Fig. 1 shows the I-CVI after first round of the Delphi survey and I-CVI after modification of the tool.
Thirty-seven HAMs were studied to evaluate the interrater reliability of K-HAMST risk scores. The scores for each medication were assessed differently at each site as shown in Table 1. The overall Kendall’s coefficient of concordance (W) for the 37 medications was 0.57, indicating moderate agreement between the raters (
Table 1 Final risk score of HAMs by utilization of K-HAMST
HAM | Category | Final | Site 1 | Site 2 | Site 3 | Site 4 | Site 5 | Site 6 | Median |
---|---|---|---|---|---|---|---|---|---|
Doxorubicin | Chemotherapeutic agents | 8 | 8 | 7 | 8 | 8 | 8 | 5 | 8 |
Rituximab | Chemotherapeutic agents | 8 | 9 | 8 | 9 | 8 | 8 | 7 | 8 |
Cisplatin | Chemotherapeutic agents | 7 | 8 | 6 | 7 | 7 | 8 | 5 | 7 |
Ketamine | Anesthetics | 7 | 6 | 7 | 7 | 8 | 7 | 6 | 7 |
Potassium chloride | Mineral supplements | 6 | 7 | 4 | 7 | 6 | 6 | 4 | 6 |
Midazolam | Benzodiazepines | 6 | 9 | 3 | 7 | 5 | 3 | 6 | 5.5 |
Morphine | Opioids | 5 | 7 | 3 | 7 | 6 | 4 | 3 | 5 |
Remifentanil | Opioids | 5 | 9 | 3 | 9 | 7 | 3 | 2 | 5 |
Heparin | Antithrombotic agents | 5 | 8 | 5 | 5 | 5 | 5 | 2 | 5 |
Regular Insulin | Insulin | 5 | 5 | 5 | 3 | 6 | 4 | 3 | 4.5 |
Fentanyl | Opioids | 5 | 7 | 3 | 8 | 5 | 4 | 3 | 4.5 |
Oxycodone | Opioids | 5 | 5 | 3 | 5 | 6 | 4 | 4 | 4.5 |
50% Magnesium sulfate | Mineral supplements | 5 | 7 | 5 | 6 | 1 | 3 | 4 | 4.5 |
Alteplase | Antithrombotic agents | 4 | 6 | 2 | 5 | 4 | 4 | 2 | 4 |
Dexmedetomidine | Hypnotics/sedatives | 4 | 6 | 3 | 4 | 5 | 4 | 3 | 4 |
Dopamine | Cardiac stimulants | 4 | 5 | 3 | 4 | 4 | 5 | 2 | 4 |
Dobutamine | Cardiac stimulants | 4 | 6 | 3 | 4 | 4 | 5 | 3 | 4 |
Hypertonic Saline | Saline | 4 | 7 | 2 | 3 | 3 | 5 | 5 | 4 |
Rocuronium | Muscle relaxants | 4 | 8 | 5 | 3 | 4 | 4 | 3 | 4 |
Propofol | Anesthetics | 4 | 7 | 3 | 8 | 5 | 3 | 3 | 4 |
Capecitabine | Chemotherapeutic agents | 4 | 4 | 6 | 4 | 4 | 3 | 4 | 4 |
Abiraterone | Chemotherapeutic agents | 4 | 4 | 4 | 4 | 2 | 3 | 4 | 4 |
Imatinib | Chemotherapeutic agents | 4 | 4 | 5 | 4 | 4 | 3 | 4 | 4 |
Lorazepam | Benzodiazepines | 4 | 5 | 3 | 6 | 3 | 3 | 5 | 4 |
Liposomal amphotericin B | Antibiotics | 4 | 6 | 4 | 3 | 3 | 4 | 3 | 3.5 |
Epinephrine | Cardiac stimulants | 3 | 6 | 3 | 3 | 4 | 2 | 3 | 3 |
Amikacin | Antibiotics | 3 | 4 | 1 | 1 | 4 | 3 | 3 | 3 |
Amiodarone | Antiarrhythmics | 3 | 5 | 3 | 3 | 3 | 6 | 2 | 3 |
Vancomycin | Antibiotics | 3 | 4 | 2 | 3 | 4 | 2 | 1 | 2.5 |
Rivaroxaban | Antithrombotic agents | 3 | 4 | 3 | 1 | 3 | 2 | 2 | 2.5 |
Enoxaparin | Antithrombotic agents | 3 | 3 | 3 | 1 | 3 | 2 | 2 | 2.5 |
Apixaban | Antithrombotic agents | 3 | 3 | 2 | 1 | 3 | 2 | 3 | 2.5 |
Warfarin | Antithrombotic agents | 3 | 3 | 4 | 2 | 3 | 2 | 2 | 2.5 |
Buprenorphine | Opioids | 3 | 4 | 2 | 5 | 2 | 3 | 2 | 2.5 |
Tacrolimus | Immunosuppressants | 3 | 4 | 3 | 2 | 2 | 4 | 2 | 2.5 |
Fosphenytoin | Antiepileptics | 3 | 6 | 2 | 3 | 2 | 4 | 1 | 2.5 |
Lidocaine | Anesthetics | 3 | 5 | 2 | 2 | 3 | 3 | 2 | 2.5 |
Six panelists rated the scores on a total of 74 mediations using the newly developed K-HAMST. The final risk scores of HAM and control medications are shown in Table 1 and 2, respectively. The HAM scores ranged from 3 to 8 (n=37; median [IQR], 4 [3–5]), while the scores of control medications ranged from 1 to 2 (n=37; median [IQR], 1 [1]). The highest scoring HAMs were IV chemotherapeutic agents, such as doxorubicin and rituximab, both with a score of 8, and cisplatin with a score of 7. The lowest scoring HAMs were epinephrine, vancomycin, fosphenytoin, apixaban, lidocaine, rivaroxaban, amiodarone, and warfarin, with scores of 3. The highest scoring controls were ondansetron, esomeprazole, and zoledronic acid, with scores of 2. The results indicated that a K-HAMST score of 3 was considered a sufficient cutoff score to identify HAMs. The K-HAMST score of 6 or higher identifies the top 25% of HAMs, indicating medications with the highest risk. Fig. 2 shows the distribution of HAMs and control medications with the newly established cutoff score.
Table 2 Final risk score of control medications by utilization of K-HAMST
Control | Category | Final | Site 1 | Site 2 | Site 3 | Site 4 | Site 5 | Site 6 | Median |
---|---|---|---|---|---|---|---|---|---|
Ondansetron | Serotonin antagonists | 2 | 2 | 3 | 1 | 1 | 3 | 2 | 2 |
Esomeprazole | Proton pump inhibitors | 2 | 2 | 3 | 1 | 1 | 2 | 2 | 2 |
Zoledronic acid | Bisphosphonates | 2 | 2 | 2 | 1 | 1 | 2 | 2 | 2 |
Acetaminophen | Analgesics | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 |
Gabapentin | Antiepileptic agents | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Influenza vaccine | Vaccines | 1 | 1 | 0 | 2 | 0 | 1 | 3 | 1 |
Amoxicillin | Antibiotics | 1 | 1 | 2 | 0 | 0 | 1 | 1 | 1 |
Bacitracin | Antibiotics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Levothyroxine | Thyroid hormones | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Ibuprofen | NSAIDs | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Aspirin | NSAIDs | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Budesonide/formoterol | Corticosteroids/adrenergic agonists | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Salbutamol | Adrenergic agonists | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Hyaluronate | Antirheumatics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Tiotropium | Anticholinergics | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 |
Aprepitant | Antiemetics | 1 | 1 | 1 | 0 | 0 | 2 | 1 | 1 |
Donepezil | Anticholinesterases | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 |
Furosemide | Diuretics | 1 | 0 | 2 | 1 | 0 | 1 | 1 | 1 |
Metformin | Antidiabetics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Polyethylene glycol | Laxatives | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Bisacodyl/docusate | Laxatives | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Ferrous sulfate | Iron supplements | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Lisinopril | ACE inhibitors | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Cetirizine | Antihistamines | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Prednisone | Corticosteroids | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
Losartan | Angiotensin receptor blockers | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Tropicamide | Anticholinergics | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Atorvastatin | Lipid modifying agents | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Diltiazem | Calcium-channel blockers | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
Famotidine | H2-receptor antagonists | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
NSAIDs=non-steroidal anti-inflammatory drugs, ACE inhibitors=angiotensin-converting enzyme inhibitor.
This is the first multi-site study to develop a high-alert medication stratification tool for acute care settings in Korea. The newly developed K-HAMST allows medical institutions to have a standardized tool to develop a HAM list with guidance to add or drop specific medications depending on the medical practice of each institution. The questions of the tool were adapted from a previous study that developed the HAMST-R, which was derived from the ASHP high-alert medication checklist [10]. In a previous study, a HAM list was created by incorporating a HAM list from six institutions. In our study, we derived a composite HAM list by including medications mentioned in the ISMP, APINCHS, and HAMST-R studies [1,7,8]. We obtained expert opinions from ten hospital pharmacists with the selection process of including frequently used medications and excluding medications that are not marketed or routinely used in Korea to create a medication list to test K-HAMST.
HAMST-R in the previous study received I-CVI scores ranging from 0.50 to 1.0 with the overall S-CVI being 0.80 [9]. The initial S-CVI score of the newly developed tool was 0.70. After the modification, we obtained an S-CVI of 0.92, indicating that the content of the tool is valid. Content validity in this study was assessed using a two-round Delphi survey process with expert opinion to validate the revised version of the K-HAMST. The survey allowed the study participants to add or modify any content of the items listed by the researchers. This may include specific medical terms or regulatory programs in Korea. We believe that the Delphi process and expert meetings allowed us to develop a more practical and applicable tool that fully reflects the opinions of experts with experience in hospital practice in Korea.
We were able to confirm the interrater reliability of the tool by testing the tool on a pre-determined HAMs list. This study also adapted the method of calculating interrater reliability using Kendall’s coefficient of concordance, similar to previous studies. Kendall’s coefficient of concordance (W) was 0.57, which suggests moderate agreement among raters (
The K-HAMST was tested by six pharmacists practicing in hospitals in Korea to determine whether the tool could successfully distinguish HAMs from control medications. The cut-off score for the HAMST-R was 4, while the application of the K-HAMST indicated a score of 3 as the cut-off score to identify HAMs. This result indicates that the cut-off score may vary between countries depending on the use of medications, practice guidelines, and hospital protocols specific to each country.
Our goal in developing K-HAMST was to create a validated tool that can identify HAMs specific to Korea. In other words, the method to develop K-HAMST can be implemented in other countries that lack standardized tools to identify HAMs. Once the tool is developed, it can be used in multiple institutions as long as they share the same clinical workflow. Each institution may have its own answer to the tool’s questions. For example, question 6-(1) “Are verbal phone orders prohibited?” was answered differently from one hospital to another in our research. This may impact whether or not the medication of interest is included in the HAMs list. Our method to develop K-HAMST allows the HAMST-R to be tailored to the clinical processes of each country and specific questions of the tool allow the HAMs to be identified according to each clinical setting.
We referred to phase 2 study of the HAMST-R by Shenk et al. [9] when determining the sample size for appropriate evaluation of content validity and interrater reliability, where content validity was evaluated by ten individuals (9 pharmacists and 1 nurse) and HAM scores were obtained from 6 different sites for interrater reliability. We have determined that 10 clinical pharmacists from six different sites may be sufficient to determine the content validity and interrater reliability of our tool. According to a review article of content validity index, minimum I-CVI of 0.78 is recommended for 6 to 10 experts [13]. Our study with 10 experts and I-CVI score of 0.79 to have content validity, therefore, was sufficient for evaluating our tool.
We have identified that with the K-HAMST tool, the highest score was 9, whereas the HAMST-R score ranged from 0 to 10. Sub-item 10-(1) evaluates if the medication has an associated Risk Management Program (RMP) requirement [18]. This corresponds to Risk Evaluation and Mitigation Strategy requirements mentioned in the HAMST-R tool. At the present time, medications that are listed in the RMP are isotretinoin, alitretinoin, acitretin, thalidomide, lenalidomide, valproic acid, anorexiants. Our sample HAMs list, however, does not include any of the drugs mentioned in the RMP. Thus, this may have affected our score distribution different from the HAMST-R study, having no score of 10.
One main limitation should be considered before adopting this tool. The scores of the medications may be inconsistent among different healthcare systems or institutions. According to the K-HAMST, ondansetron and esomeprazole received a risk score of 2, indicating that the two medications are not considered HAMs. However, two study institutions ranked ondansetron with a score of 3, and one institution ranked esomeprazole with a score of 3. Thus, for some institutions, ondansetron and esomeprazole are considered HAMs. This result shows that there is a possibility of discrepancies among different healthcare systems when evaluating the same medications.
Additional evaluation of ease of use regarding any barriers while implementing the tool in actual practice may be necessary. Further prospective studies on the utilization of the tool by institutions in Korea that have not yet developed an objective formulary may also be considered valuable.
This study developed the K-HAMST and evaluated it was a valid and reliable tool for use in the hospital setting. The results indicated that a K-HAMST score 3 or higher identified HAMs. The K-HAMST may be implemented in clinical settings to identify new HAMs or modify existing HAM lists.
This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI21C1389).
None.
No potential conflict of interest relevant to this article was reported.
No potential conflict of interest relevant to this article was reported.
Supplementary materials can be found via https://doi.org/10.59931/rcp.23.005.
rcp-1-1-40-supple.pdfR Clin Pharm 2023; 1(1): 40-48
Published online June 30, 2023 https://doi.org/10.59931/rcp.23.005
Copyright © Asian Conference On Clinical Pharmacy.
Ahyoung Lee , Ji Min Han , Kwanghee Jun , Kyu Nam Heo , McKenzie S. Grinalds , Natalie C. Washburn , Todd A. Walroth , Young-Mi Ah , Ju-Yeun Lee
College of Pharmacy, Seoul National University, Seoul, Korea
Correspondence to:Ju-Yeun Lee
E-mail jypharm@snu.ac.kr
ORCID
https://orcid.org/0000-0002-2261-7330
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/bync/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background: There is no objective and standardized tool for stratification of high-alert medications (HAMs) that reflect each institution’s practice of medication management, drug utilization in the institution, and patient safety protocols in the Korean hospital settings. We aimed to develop the Korean version of the high-alert medication stratification tool (K-HAMST) and assess its content validity and reliability.
Methods: Ten clinical pharmacists from multi-site hospitals completed a two-round Delphi survey to assess the content validity of the translated High-Alert Medication Stratification Tool-Revised. Content validity was demonstrated using the item content validity index and scale content validity index (S-CVI). An expert meeting was conducted to revise the tool to accommodate the clinical practice and workflow in Korea based on the results of the content validity index. Reliability was assessed by calculating the risk scores for 37 HAMs and 37 control medications. The interrater reliability of each medication was assessed using the Kendall’s coefficient of concordance (W).
Results: The initial S-CVI was 0.71. After revision, the final S-CVI of the K-HAMST was 0.92, indicating that the tool has content validity. The HAM scores ranged from 3 to 8 (n=37; median [interquartile {IQR}, 4 [3–5]), whereas the control medication scores ranged from 1 to 2 (n=37; median [IQR], 1 [1–1]). The Kendall’s coefficient of concordance (W) was 0.57, indicating moderate agreement between raters (p<0.001).
Conclusion: The K-HAMST is a valid and reliable tool for assessing and evaluating HAMs in hospital settings.
Keywords: Content validity, HAM identification, High-alert medication, Medication safety, Risk stratification tool
The Institute for Safe Medication Practices (ISMP) defines “high-alert medications (HAMs) as drugs that bear a heightened risk of causing significant patient harm when used in error” [1]. According to a review of events in a database of 317 preventable adverse drug events (ADEs), approximately 60% of the preventable ADEs resulted from the use of HAMs [2]. In a trigger-based ADE identification study conducted in Spain, 74.5% of all ADEs identified in geriatric patients with multimorbidity were attributed to HAMs [3]. As previous studies have shown [4,5], the consequences of errors with HAMs are more devastating to patients since they contribute to a substantial number of preventable ADEs compared to other medications.
HAMs are directly associated with patient safety risks in the healthcare system, which is a strong trigger for each country to develop HAM lists and management guidelines. Previously, researchers reported HAM lists and tools for the identification of such medications [6-9]. In 2015, to support the development and modification of an institutional HAM list, the American Society of Health-System Pharmacists (ASHP) Inpatient Care Practitioner’s Medication Safety Advisory Group distributed an objective high-alert medication checklist. The purpose of the ASHP checklist is to create or modify each organization’s HAM list to meet the needs of a specific institution [10].
Eskenazi Health in the United States developed a standardized high-alert medication stratification tool (HAMST-R) based on the aforementioned ASHP checklist after subsequent validation and interrater reliability testing. According to the HAMST-R study, the HASMT-R consists of 10 items; items 1–4 evaluate overall patient safety issues while items 5–10 evaluate safety concerns in the steps of the drug use process. If more than 50% of the questions within each step of the drug use process are answered “yes”, 1 point is added for that specific phase. The total score adds up to 10 when all phases receive a point, and a HAMST-R score of 4 or higher identifies a HAM [8]. The HAMST-R, developed in the United States, was shown to be a valid and reliable tool for evaluating medication risks in hospital settings across the United States and Canada [8,9].
In Korea, there is no objective and standardized tool for stratification of HAMs in hospital settings. The Korea Institute for Healthcare Accreditation (KOIHA) suggests HAM categories including sedative drugs, anticancer drugs, high-concentration electrolytes, antithrombotic agents for injection, insulin, and contrast agents based on expert opinion [11]. These Korean-specific HAM categories are insufficient to reflect each institution’s practice of medication management, drug utilization in the institution, and patient safety protocols. Among the numerous tools to identify HAMs, the HAMST-R is a structured and standardized risk stratification tool. We adjusted HAMST-R, known as HAMST-R PRO, to be used prospectively and published the results [12]. Furthermore, the tool can be used for analyses of safety practices of HAMs by evaluating the medication use process of prescribing, transcribing, dispensing, and administration as outlined in questions 5 to 10 of HAMST-R [8,9]. A heightened emphasis on the management of HAMs and insufficient HAMs identification tools in Korea promotes the development of an objective and quantitative tool that can be utilized across the healthcare systems in Korea.
In this study, we developed the K-HAMST based on Washburn et al. [8] study of the HAMST-R and tested the validity and interrater reliability of K-HAMST by referring to the phase 2 study of the HAMST-R by Shenk et al. [9]. The following steps were conducted: 1) translation and back-translation, 2) content validity testing using the Delphi method, and 3) interrater reliability testing using the K-HAMST.
An advisory group consisting of ten clinical pharmacists from medical institutions was recommended by the Patient Safety Quality Improvement Committee under the Korean Society of Health-System Pharmacists (KSHP). All the recommended pharmacists had more than 5 years of experience in hospital pharmacy. Of the ten pharmacists, interrater reliability verification was conducted by six clinical pharmacists from six independent tertiary hospitals in Korea.
In this study, the translation process was established using translation, back-translation, and proofreading performed by the researchers. Back-translation is a well-known method for maintaining equivalence between original and translated versions of tools and questionnaires [13]. The HAMST-R was translated into Korean by two bilingual researchers conducting this study. Back-translation was performed by translating the Korean version back into English without referring to the original tool. During the proofreading process, the back-translated English version of the tool was compared to HAMST-R to validate whether all meanings of the subitems were translated equivalently.
Content validity refers to the method of estimating the validity of a new or revised tool quantitatively. A widely used method is known as the content validity index, which is based on expert rating for each item of the scale. An item content validity index (I-CVI) represents the validity of an item on a scale, whereas the scale content validity index (S-CVI) represents the validity of the overall scale [14]. Ten clinical pharmacists completed a two-round Delphi survey to assess the content validity of the K-HAMST translated and adapted from the HAMST-R. The study participants provided consent to enroll in the two-round Delphi process. An initial online Delphi survey to evaluate content validity was conducted on the 30 sub-items of the K-HAMST. The online survey consisted of 30 questions to assess the validity of the tool with an ordinal scale from 1 to 4, allowing the participants to provide opinions on each item. The 4-point ordinal scale denotes 1 (not relevant), 2 (somewhat relevant), 3 (quite relevant), and 4 (highly relevant). If an I-CVI was higher than 79 percent, the item was appropriate, and if it was between 70 and 79 percent, the item required revision [15]. The tool was considered to have content validity if the overall S-CVI was higher than 0.80 [16].
If the I-CVI was less than 70 percent, we revised the question instead of being eliminated. After the first online survey, an expert meeting was conducted involving ten study participants to revise the tool to accommodate the medical situation and drug management protocols in Korea by focusing on questions with an I-CVI of less than 0.79. For the items that received a low I-CVI of less than 0.79, the group went through an extensive discussion to modify items or add examples to each item. The modified items specific to medical practices in Korea may guide the evaluators to easily assess and confirm whether the medication in interest is considered HAM or not. The second Delphi survey to test the final validation of the revised questions using content validity rating was conducted with a similar procedure as the first survey.
The consistency or agreement among the experts interpreting the data is important. The extent of agreement among data collectors is called, “
We used a sample size of 37 for each group, namely the HAMs and controls, to obtain 80% power as done in the previous study of the HAMST-R [8]. A comprehensive list of HAMs and control medications (non-HAMs) was derived from composite list of HAMs from the ISMP, Australian Commission on Safety and Quality in Health Care (APINCHS), and the HAMST-R study [1,7,8]. We have excluded medications that are not marketed or routinely used in Korea from this composite list and we finalized the list with the expert opinion of 10 clinical pharmacists from six different hospitals, considering both the frequency of use in hospitals and the extent of harm to the patient. Our list was used to determine the interrater reliability of the K-HAMST and to determine a cut-off score specific for use in hospitals in Korea. The interrater reliability of the K-HAMST was studied by six clinical pharmacists selected to represent six different institutions. Reliability was assessed by calculating the risk scores for 37 HAMs and 37 control medications. Prior to the utilization of the tool, the researchers provided the participants with instructions on how to answer each question and provided a list of HAMs and control medications, including specific formulations and product brand names. The study participants evaluated 74 medications and rated each medication based on national and local hospital guidelines for the management of medications. The K-HAMST questions were scored on four general patient safety concerns and six phases of the medication use process, similar to the previously developed HAMST-R. Similar to the previous study, each phase consisted of a different number of questions. Directions stated that if more than 50% of the questions within each phase of the medication use process were answered “yes”, 1 point was given for the specific phase. The maximum total score was 10 when all phases received a score [8,17].
The Anderson–Darling test was performed to test the normality of the data. The significance of variance between data was determined using the Mann-Whitney U test with an alpha value of less than 0.05. In this study, I-CVI was computed as the number of experts giving a rating 3 or 4 to the relevancy of each item, divided by the total number of experts. The S-CVI was the average of the I-CVI score [16]. The final score was the median of the scores evaluated for each site, and the median score was rounded up to the nearest whole number. The threshold score for the I-CVI was less than 0.79, indicating that the content of the item was not relevant and needed modification. To measure how consistently the raters evaluated each medication, the interrater reliability of the risk scores of each medication was studied using the Kendall’s coefficient of concordance (W) method. The value of W ≤0 indicates no agreement, 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement [17]. The cutoff score to determine HAMs was established based on the risk scores of the HAMs and control medications (N=74) evaluated by six hospital pharmacists. The threshold was based on a score that included only HAMs and no control medications. Statistical analysis was performed using the SPSS version 22.0 software (IBM Corporation, Armonk, NY, USA).
The overview of the final version of K-HAMST in English is shown in Supplementary Table 1. The results of the translation of 30 questions are shown in Supplementary Table 2. Based on the results of the first-round Delphi survey, modification of the questions and addition of references or examples was done to add clarification to each question. The results of the modified K-HAMST that reflect expert opinion are shown in Supplementary Table 3.
During the first Delphi survey, the I-CVI scores ranged from 0.4–0.9, and 14 of 30 items that received an I-CVI greater than 0.79 met the criteria for significance (Fig. 1). The overall validity of the tool after the first round of the Delphi survey was 0.71, indicating that the S-CVI of the scale before revision did not have content validity since it was less than 0.80 [16]. Of the total of 30 questions, 16 questions that received I-CVI less than 0.79 went through the modification process. After the revision of the tool, the content validity of the K-HAMST was re-evaluated using the second Delphi survey. All 30 items achieved an I-CVI greater than 0.79, with scores that ranged from 0.8–1.0. The S-CVI score increased to 0.92, indicating that the tool had content validity. Fig. 1 shows the I-CVI after first round of the Delphi survey and I-CVI after modification of the tool.
Thirty-seven HAMs were studied to evaluate the interrater reliability of K-HAMST risk scores. The scores for each medication were assessed differently at each site as shown in Table 1. The overall Kendall’s coefficient of concordance (W) for the 37 medications was 0.57, indicating moderate agreement between the raters (
Table 1 . Final risk score of HAMs by utilization of K-HAMST.
HAM | Category | Final | Site 1 | Site 2 | Site 3 | Site 4 | Site 5 | Site 6 | Median |
---|---|---|---|---|---|---|---|---|---|
Doxorubicin | Chemotherapeutic agents | 8 | 8 | 7 | 8 | 8 | 8 | 5 | 8 |
Rituximab | Chemotherapeutic agents | 8 | 9 | 8 | 9 | 8 | 8 | 7 | 8 |
Cisplatin | Chemotherapeutic agents | 7 | 8 | 6 | 7 | 7 | 8 | 5 | 7 |
Ketamine | Anesthetics | 7 | 6 | 7 | 7 | 8 | 7 | 6 | 7 |
Potassium chloride | Mineral supplements | 6 | 7 | 4 | 7 | 6 | 6 | 4 | 6 |
Midazolam | Benzodiazepines | 6 | 9 | 3 | 7 | 5 | 3 | 6 | 5.5 |
Morphine | Opioids | 5 | 7 | 3 | 7 | 6 | 4 | 3 | 5 |
Remifentanil | Opioids | 5 | 9 | 3 | 9 | 7 | 3 | 2 | 5 |
Heparin | Antithrombotic agents | 5 | 8 | 5 | 5 | 5 | 5 | 2 | 5 |
Regular Insulin | Insulin | 5 | 5 | 5 | 3 | 6 | 4 | 3 | 4.5 |
Fentanyl | Opioids | 5 | 7 | 3 | 8 | 5 | 4 | 3 | 4.5 |
Oxycodone | Opioids | 5 | 5 | 3 | 5 | 6 | 4 | 4 | 4.5 |
50% Magnesium sulfate | Mineral supplements | 5 | 7 | 5 | 6 | 1 | 3 | 4 | 4.5 |
Alteplase | Antithrombotic agents | 4 | 6 | 2 | 5 | 4 | 4 | 2 | 4 |
Dexmedetomidine | Hypnotics/sedatives | 4 | 6 | 3 | 4 | 5 | 4 | 3 | 4 |
Dopamine | Cardiac stimulants | 4 | 5 | 3 | 4 | 4 | 5 | 2 | 4 |
Dobutamine | Cardiac stimulants | 4 | 6 | 3 | 4 | 4 | 5 | 3 | 4 |
Hypertonic Saline | Saline | 4 | 7 | 2 | 3 | 3 | 5 | 5 | 4 |
Rocuronium | Muscle relaxants | 4 | 8 | 5 | 3 | 4 | 4 | 3 | 4 |
Propofol | Anesthetics | 4 | 7 | 3 | 8 | 5 | 3 | 3 | 4 |
Capecitabine | Chemotherapeutic agents | 4 | 4 | 6 | 4 | 4 | 3 | 4 | 4 |
Abiraterone | Chemotherapeutic agents | 4 | 4 | 4 | 4 | 2 | 3 | 4 | 4 |
Imatinib | Chemotherapeutic agents | 4 | 4 | 5 | 4 | 4 | 3 | 4 | 4 |
Lorazepam | Benzodiazepines | 4 | 5 | 3 | 6 | 3 | 3 | 5 | 4 |
Liposomal amphotericin B | Antibiotics | 4 | 6 | 4 | 3 | 3 | 4 | 3 | 3.5 |
Epinephrine | Cardiac stimulants | 3 | 6 | 3 | 3 | 4 | 2 | 3 | 3 |
Amikacin | Antibiotics | 3 | 4 | 1 | 1 | 4 | 3 | 3 | 3 |
Amiodarone | Antiarrhythmics | 3 | 5 | 3 | 3 | 3 | 6 | 2 | 3 |
Vancomycin | Antibiotics | 3 | 4 | 2 | 3 | 4 | 2 | 1 | 2.5 |
Rivaroxaban | Antithrombotic agents | 3 | 4 | 3 | 1 | 3 | 2 | 2 | 2.5 |
Enoxaparin | Antithrombotic agents | 3 | 3 | 3 | 1 | 3 | 2 | 2 | 2.5 |
Apixaban | Antithrombotic agents | 3 | 3 | 2 | 1 | 3 | 2 | 3 | 2.5 |
Warfarin | Antithrombotic agents | 3 | 3 | 4 | 2 | 3 | 2 | 2 | 2.5 |
Buprenorphine | Opioids | 3 | 4 | 2 | 5 | 2 | 3 | 2 | 2.5 |
Tacrolimus | Immunosuppressants | 3 | 4 | 3 | 2 | 2 | 4 | 2 | 2.5 |
Fosphenytoin | Antiepileptics | 3 | 6 | 2 | 3 | 2 | 4 | 1 | 2.5 |
Lidocaine | Anesthetics | 3 | 5 | 2 | 2 | 3 | 3 | 2 | 2.5 |
Six panelists rated the scores on a total of 74 mediations using the newly developed K-HAMST. The final risk scores of HAM and control medications are shown in Table 1 and 2, respectively. The HAM scores ranged from 3 to 8 (n=37; median [IQR], 4 [3–5]), while the scores of control medications ranged from 1 to 2 (n=37; median [IQR], 1 [1]). The highest scoring HAMs were IV chemotherapeutic agents, such as doxorubicin and rituximab, both with a score of 8, and cisplatin with a score of 7. The lowest scoring HAMs were epinephrine, vancomycin, fosphenytoin, apixaban, lidocaine, rivaroxaban, amiodarone, and warfarin, with scores of 3. The highest scoring controls were ondansetron, esomeprazole, and zoledronic acid, with scores of 2. The results indicated that a K-HAMST score of 3 was considered a sufficient cutoff score to identify HAMs. The K-HAMST score of 6 or higher identifies the top 25% of HAMs, indicating medications with the highest risk. Fig. 2 shows the distribution of HAMs and control medications with the newly established cutoff score.
Table 2 . Final risk score of control medications by utilization of K-HAMST.
Control | Category | Final | Site 1 | Site 2 | Site 3 | Site 4 | Site 5 | Site 6 | Median |
---|---|---|---|---|---|---|---|---|---|
Ondansetron | Serotonin antagonists | 2 | 2 | 3 | 1 | 1 | 3 | 2 | 2 |
Esomeprazole | Proton pump inhibitors | 2 | 2 | 3 | 1 | 1 | 2 | 2 | 2 |
Zoledronic acid | Bisphosphonates | 2 | 2 | 2 | 1 | 1 | 2 | 2 | 2 |
Acetaminophen | Analgesics | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 |
Gabapentin | Antiepileptic agents | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Influenza vaccine | Vaccines | 1 | 1 | 0 | 2 | 0 | 1 | 3 | 1 |
Amoxicillin | Antibiotics | 1 | 1 | 2 | 0 | 0 | 1 | 1 | 1 |
Bacitracin | Antibiotics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Levothyroxine | Thyroid hormones | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Ibuprofen | NSAIDs | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Aspirin | NSAIDs | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Budesonide/formoterol | Corticosteroids/adrenergic agonists | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Salbutamol | Adrenergic agonists | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Hyaluronate | Antirheumatics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Tiotropium | Anticholinergics | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 |
Aprepitant | Antiemetics | 1 | 1 | 1 | 0 | 0 | 2 | 1 | 1 |
Donepezil | Anticholinesterases | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 |
Furosemide | Diuretics | 1 | 0 | 2 | 1 | 0 | 1 | 1 | 1 |
Metformin | Antidiabetics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Polyethylene glycol | Laxatives | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Bisacodyl/docusate | Laxatives | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Ferrous sulfate | Iron supplements | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Lisinopril | ACE inhibitors | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Cetirizine | Antihistamines | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Prednisone | Corticosteroids | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
Losartan | Angiotensin receptor blockers | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Tropicamide | Anticholinergics | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Atorvastatin | Lipid modifying agents | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Diltiazem | Calcium-channel blockers | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
Famotidine | H2-receptor antagonists | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
NSAIDs=non-steroidal anti-inflammatory drugs, ACE inhibitors=angiotensin-converting enzyme inhibitor..
This is the first multi-site study to develop a high-alert medication stratification tool for acute care settings in Korea. The newly developed K-HAMST allows medical institutions to have a standardized tool to develop a HAM list with guidance to add or drop specific medications depending on the medical practice of each institution. The questions of the tool were adapted from a previous study that developed the HAMST-R, which was derived from the ASHP high-alert medication checklist [10]. In a previous study, a HAM list was created by incorporating a HAM list from six institutions. In our study, we derived a composite HAM list by including medications mentioned in the ISMP, APINCHS, and HAMST-R studies [1,7,8]. We obtained expert opinions from ten hospital pharmacists with the selection process of including frequently used medications and excluding medications that are not marketed or routinely used in Korea to create a medication list to test K-HAMST.
HAMST-R in the previous study received I-CVI scores ranging from 0.50 to 1.0 with the overall S-CVI being 0.80 [9]. The initial S-CVI score of the newly developed tool was 0.70. After the modification, we obtained an S-CVI of 0.92, indicating that the content of the tool is valid. Content validity in this study was assessed using a two-round Delphi survey process with expert opinion to validate the revised version of the K-HAMST. The survey allowed the study participants to add or modify any content of the items listed by the researchers. This may include specific medical terms or regulatory programs in Korea. We believe that the Delphi process and expert meetings allowed us to develop a more practical and applicable tool that fully reflects the opinions of experts with experience in hospital practice in Korea.
We were able to confirm the interrater reliability of the tool by testing the tool on a pre-determined HAMs list. This study also adapted the method of calculating interrater reliability using Kendall’s coefficient of concordance, similar to previous studies. Kendall’s coefficient of concordance (W) was 0.57, which suggests moderate agreement among raters (
The K-HAMST was tested by six pharmacists practicing in hospitals in Korea to determine whether the tool could successfully distinguish HAMs from control medications. The cut-off score for the HAMST-R was 4, while the application of the K-HAMST indicated a score of 3 as the cut-off score to identify HAMs. This result indicates that the cut-off score may vary between countries depending on the use of medications, practice guidelines, and hospital protocols specific to each country.
Our goal in developing K-HAMST was to create a validated tool that can identify HAMs specific to Korea. In other words, the method to develop K-HAMST can be implemented in other countries that lack standardized tools to identify HAMs. Once the tool is developed, it can be used in multiple institutions as long as they share the same clinical workflow. Each institution may have its own answer to the tool’s questions. For example, question 6-(1) “Are verbal phone orders prohibited?” was answered differently from one hospital to another in our research. This may impact whether or not the medication of interest is included in the HAMs list. Our method to develop K-HAMST allows the HAMST-R to be tailored to the clinical processes of each country and specific questions of the tool allow the HAMs to be identified according to each clinical setting.
We referred to phase 2 study of the HAMST-R by Shenk et al. [9] when determining the sample size for appropriate evaluation of content validity and interrater reliability, where content validity was evaluated by ten individuals (9 pharmacists and 1 nurse) and HAM scores were obtained from 6 different sites for interrater reliability. We have determined that 10 clinical pharmacists from six different sites may be sufficient to determine the content validity and interrater reliability of our tool. According to a review article of content validity index, minimum I-CVI of 0.78 is recommended for 6 to 10 experts [13]. Our study with 10 experts and I-CVI score of 0.79 to have content validity, therefore, was sufficient for evaluating our tool.
We have identified that with the K-HAMST tool, the highest score was 9, whereas the HAMST-R score ranged from 0 to 10. Sub-item 10-(1) evaluates if the medication has an associated Risk Management Program (RMP) requirement [18]. This corresponds to Risk Evaluation and Mitigation Strategy requirements mentioned in the HAMST-R tool. At the present time, medications that are listed in the RMP are isotretinoin, alitretinoin, acitretin, thalidomide, lenalidomide, valproic acid, anorexiants. Our sample HAMs list, however, does not include any of the drugs mentioned in the RMP. Thus, this may have affected our score distribution different from the HAMST-R study, having no score of 10.
One main limitation should be considered before adopting this tool. The scores of the medications may be inconsistent among different healthcare systems or institutions. According to the K-HAMST, ondansetron and esomeprazole received a risk score of 2, indicating that the two medications are not considered HAMs. However, two study institutions ranked ondansetron with a score of 3, and one institution ranked esomeprazole with a score of 3. Thus, for some institutions, ondansetron and esomeprazole are considered HAMs. This result shows that there is a possibility of discrepancies among different healthcare systems when evaluating the same medications.
Additional evaluation of ease of use regarding any barriers while implementing the tool in actual practice may be necessary. Further prospective studies on the utilization of the tool by institutions in Korea that have not yet developed an objective formulary may also be considered valuable.
This study developed the K-HAMST and evaluated it was a valid and reliable tool for use in the hospital setting. The results indicated that a K-HAMST score 3 or higher identified HAMs. The K-HAMST may be implemented in clinical settings to identify new HAMs or modify existing HAM lists.
This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI21C1389).
None.
No potential conflict of interest relevant to this article was reported.
No potential conflict of interest relevant to this article was reported.
Supplementary materials can be found via https://doi.org/10.59931/rcp.23.005.
rcp-1-1-40-supple.pdfTable 1 Final risk score of HAMs by utilization of K-HAMST
HAM | Category | Final | Site 1 | Site 2 | Site 3 | Site 4 | Site 5 | Site 6 | Median |
---|---|---|---|---|---|---|---|---|---|
Doxorubicin | Chemotherapeutic agents | 8 | 8 | 7 | 8 | 8 | 8 | 5 | 8 |
Rituximab | Chemotherapeutic agents | 8 | 9 | 8 | 9 | 8 | 8 | 7 | 8 |
Cisplatin | Chemotherapeutic agents | 7 | 8 | 6 | 7 | 7 | 8 | 5 | 7 |
Ketamine | Anesthetics | 7 | 6 | 7 | 7 | 8 | 7 | 6 | 7 |
Potassium chloride | Mineral supplements | 6 | 7 | 4 | 7 | 6 | 6 | 4 | 6 |
Midazolam | Benzodiazepines | 6 | 9 | 3 | 7 | 5 | 3 | 6 | 5.5 |
Morphine | Opioids | 5 | 7 | 3 | 7 | 6 | 4 | 3 | 5 |
Remifentanil | Opioids | 5 | 9 | 3 | 9 | 7 | 3 | 2 | 5 |
Heparin | Antithrombotic agents | 5 | 8 | 5 | 5 | 5 | 5 | 2 | 5 |
Regular Insulin | Insulin | 5 | 5 | 5 | 3 | 6 | 4 | 3 | 4.5 |
Fentanyl | Opioids | 5 | 7 | 3 | 8 | 5 | 4 | 3 | 4.5 |
Oxycodone | Opioids | 5 | 5 | 3 | 5 | 6 | 4 | 4 | 4.5 |
50% Magnesium sulfate | Mineral supplements | 5 | 7 | 5 | 6 | 1 | 3 | 4 | 4.5 |
Alteplase | Antithrombotic agents | 4 | 6 | 2 | 5 | 4 | 4 | 2 | 4 |
Dexmedetomidine | Hypnotics/sedatives | 4 | 6 | 3 | 4 | 5 | 4 | 3 | 4 |
Dopamine | Cardiac stimulants | 4 | 5 | 3 | 4 | 4 | 5 | 2 | 4 |
Dobutamine | Cardiac stimulants | 4 | 6 | 3 | 4 | 4 | 5 | 3 | 4 |
Hypertonic Saline | Saline | 4 | 7 | 2 | 3 | 3 | 5 | 5 | 4 |
Rocuronium | Muscle relaxants | 4 | 8 | 5 | 3 | 4 | 4 | 3 | 4 |
Propofol | Anesthetics | 4 | 7 | 3 | 8 | 5 | 3 | 3 | 4 |
Capecitabine | Chemotherapeutic agents | 4 | 4 | 6 | 4 | 4 | 3 | 4 | 4 |
Abiraterone | Chemotherapeutic agents | 4 | 4 | 4 | 4 | 2 | 3 | 4 | 4 |
Imatinib | Chemotherapeutic agents | 4 | 4 | 5 | 4 | 4 | 3 | 4 | 4 |
Lorazepam | Benzodiazepines | 4 | 5 | 3 | 6 | 3 | 3 | 5 | 4 |
Liposomal amphotericin B | Antibiotics | 4 | 6 | 4 | 3 | 3 | 4 | 3 | 3.5 |
Epinephrine | Cardiac stimulants | 3 | 6 | 3 | 3 | 4 | 2 | 3 | 3 |
Amikacin | Antibiotics | 3 | 4 | 1 | 1 | 4 | 3 | 3 | 3 |
Amiodarone | Antiarrhythmics | 3 | 5 | 3 | 3 | 3 | 6 | 2 | 3 |
Vancomycin | Antibiotics | 3 | 4 | 2 | 3 | 4 | 2 | 1 | 2.5 |
Rivaroxaban | Antithrombotic agents | 3 | 4 | 3 | 1 | 3 | 2 | 2 | 2.5 |
Enoxaparin | Antithrombotic agents | 3 | 3 | 3 | 1 | 3 | 2 | 2 | 2.5 |
Apixaban | Antithrombotic agents | 3 | 3 | 2 | 1 | 3 | 2 | 3 | 2.5 |
Warfarin | Antithrombotic agents | 3 | 3 | 4 | 2 | 3 | 2 | 2 | 2.5 |
Buprenorphine | Opioids | 3 | 4 | 2 | 5 | 2 | 3 | 2 | 2.5 |
Tacrolimus | Immunosuppressants | 3 | 4 | 3 | 2 | 2 | 4 | 2 | 2.5 |
Fosphenytoin | Antiepileptics | 3 | 6 | 2 | 3 | 2 | 4 | 1 | 2.5 |
Lidocaine | Anesthetics | 3 | 5 | 2 | 2 | 3 | 3 | 2 | 2.5 |
Table 2 Final risk score of control medications by utilization of K-HAMST
Control | Category | Final | Site 1 | Site 2 | Site 3 | Site 4 | Site 5 | Site 6 | Median |
---|---|---|---|---|---|---|---|---|---|
Ondansetron | Serotonin antagonists | 2 | 2 | 3 | 1 | 1 | 3 | 2 | 2 |
Esomeprazole | Proton pump inhibitors | 2 | 2 | 3 | 1 | 1 | 2 | 2 | 2 |
Zoledronic acid | Bisphosphonates | 2 | 2 | 2 | 1 | 1 | 2 | 2 | 2 |
Acetaminophen | Analgesics | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 |
Gabapentin | Antiepileptic agents | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Influenza vaccine | Vaccines | 1 | 1 | 0 | 2 | 0 | 1 | 3 | 1 |
Amoxicillin | Antibiotics | 1 | 1 | 2 | 0 | 0 | 1 | 1 | 1 |
Bacitracin | Antibiotics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Levothyroxine | Thyroid hormones | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Ibuprofen | NSAIDs | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Aspirin | NSAIDs | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Budesonide/formoterol | Corticosteroids/adrenergic agonists | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Salbutamol | Adrenergic agonists | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 |
Hyaluronate | Antirheumatics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Tiotropium | Anticholinergics | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 |
Aprepitant | Antiemetics | 1 | 1 | 1 | 0 | 0 | 2 | 1 | 1 |
Donepezil | Anticholinesterases | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 |
Furosemide | Diuretics | 1 | 0 | 2 | 1 | 0 | 1 | 1 | 1 |
Metformin | Antidiabetics | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
Polyethylene glycol | Laxatives | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Bisacodyl/docusate | Laxatives | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Ferrous sulfate | Iron supplements | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Lisinopril | ACE inhibitors | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.5 |
Cetirizine | Antihistamines | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Prednisone | Corticosteroids | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
Losartan | Angiotensin receptor blockers | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Tropicamide | Anticholinergics | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Atorvastatin | Lipid modifying agents | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0.5 |
Diltiazem | Calcium-channel blockers | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
Famotidine | H2-receptor antagonists | 1 | 0 | 2 | 0 | 0 | 1 | 1 | 0.5 |
NSAIDs=non-steroidal anti-inflammatory drugs, ACE inhibitors=angiotensin-converting enzyme inhibitor.