The Inter-rater Variability of Clinical Assessment in Post-anoxic Myoclonus
Background: Acute post-anoxic myoclonus (PAM) can be divided into an unfavorable (generalized/subcortical) and more favorable ((multi)focal/cortical) outcome group that could support prognostication in post-anoxic encephalopathy; however, the inter-rater variability of clinically assessing these PAM subtypes is unknown.
Methods: We prospectively examined PAM patients using a standardized video protocol. Videos were rated by three neurologists who classified PAM phenotype (generalized/(multi)focal), stimulus sensitivity, localization (proximal/distal/both), and severity (Clinical Global Impression-Severity Scale (CGI-S) and Unified Myoclonus Rating Scale (UMRS)).
Results: Poor inter-rater agreement was found for phenotype and stimulus sensitivity (κ = –0.05), moderate agreement for localization (κ = 0.46). Substantial agreement was obtained for the CGI-S (intraclass correlation coefficient (ICC) = 0.64) and almost perfect agreement for the UMRS (ICC = 0.82).
Discussion: Clinical assessment of PAM is not reproducible between physicians, and should therefore not be used for prognostication. PAM severity measured by the UMRS appears to be reliable; however, the relation between PAM severity and outcome is unknown.
Keywords: Myoclonus, post-anoxic encephalopathy, prognosis, clinical neurology examination, critical care
Citation: van Zijl JC, Beudel M, Elting JJ, de Jong BM, van der Naalt J, van den Bergh WM, et al. The inter-rater variability of clinical assessment in post-anoxic myoclonus. Tremor Other Hyperkinet Mov. 2017; 7. doi: 10.7916/D81R6XBV
*To whom correspondence should be addressed. E-mail: firstname.lastname@example.org
Editor: Elan D. Louis, Yale University, USA
Received: April 12, 2017 Accepted: June 1, 2017 Published: July 13, 2017
Copyright: © 2017 van Zijl et al. This is an open-access article distributed under the terms of the Creative Commons Attribution–Noncommercial–No Derivatives License, which permits the user to copy, distribute, and transmit the work provided that the original authors and source are credited; that no commercial use is made of the work; and that the work is not altered or transformed.
Funding: Dr. van Zijl received the MD/PhD scholarship from the University of Groningen for research purposes. Dr. Beudel received funding from the Dutch Brain Foundation. Dr. Rossetti is supported by The Swiss National Science Foundation (CR3213_143780), and received unrestricted research support from SAGE Therapeutics and UCB Pharma.
Financial Disclosures: None.
Conflict of Interest: The authors report no conflict of interest.
Ethics Statement: This study was performed in accordance with the ethical standards detailed in the Declaration of Helsinki. The authors' institutional ethics committee has approved this study. All patients that appear on video have provided written informed consent; authorization for the videotaping and for publication of the videotape was provided.
The presence of myoclonus in the first few days after anoxic brain injury has traditionally been associated with an unfavorable outcome in patients with post-anoxic encephalopathy (PAE).1,2 This acute post-anoxic myoclonus (PAM) differs in clinical characteristics, prognosis and treatment from the Lance–Adams syndrome, in which myoclonus usually occurs a few weeks after the anoxic event.2
Nowadays, in the era of target temperature management, 9–14% of these PAM patients survive.3–5 The current incidence of PAM in PAE is estimated to be 20%.3,4 In the literature, a PAM-subtype-specific prognosis has been reported: (multi)focal PAM had a better chance of good functional outcome (17–18%) than PAE patients without PAM. In contrast, patients with generalized PAM only had a favorable outcome in 3–6%.1,4–6 This might implicate that the PAM subtype can be supportive in PAE prognostication.7–9
(Multi)focal PAM consists of subtle asynchronous jerks involving the distal musculature and is considered to have a cortical origin.4,10 Stimulus sensitivity is more often present in the cortical than the subcortical myoclonus.10 Generalized PAM (also referred to as status myoclonus, myoclonus status epilepticus, or reticular reflex myoclonus) is characterized by generalized synchronous jerks with preferential involvement of proximal musculature, and is considered to have a subcortical origin.4,6,10 Despite these well-defined differences, the reliability of clinically differentiating between these PAM subtypes is unknown. We investigated this in the current study.
We prospectively included 10 PAM patients from the intensive care units of the University Medical Center Groningen, and the Lausanne University Hospital. The institutional review board of both hospitals approved the study. For all patients informed consent was obtained from a legal representative. Exclusion criteria were age <18 years, PAM onset >72 hours, GCS (Glasgow Coma Scale) >8, history of myoclonus, traumatic brain injury, epilepsy, possible explanation myoclonus other than PAE, and medication overdose.
A standardized high-definition video protocol for examination of PAM was conducted after first appearance of PAM. The video protocol consisted of recording 60 seconds of rest, 30 seconds of close-up recording of the face, and recording 32 lateralized and non-lateralized stimuli: pain stimulus of the orbit, trapezius muscle and nail bed; pin prick of the cheek, forearm, and foot; pupillary light response; corneal and glabellar reflex, biceps and knee tendon reflexes; light touch of the hand and foot; flick of the finger and toe; visual and auditory threat; jaw and nose tapping. The video protocol is available as supplementary material (Supplement A), and an example examination is provided in Video 1.
Example Examination of a Post-anoxic Myoclonus Patient. The (abridged) systematic examination of post-anoxic myoclonus (PAM) Case 2 (Table 2) and the video protocol of this study (Supplement A). Case 2 displays slight myoclonus in rest, but it seems to increase in frequency and severity after the application of stimuli. This patient is the only PAM case of this cohort that survived and recovered with only mild cognitive deficits after 6 months.
Videos were rated by three experienced (intensive-care) neurologists (B.M.J., J.N., J.H.) blinded for outcome. Raters classified PAM phenotype (generalized or (multi)focal), localization (proximal, distal, or both), stimulus sensitivity (present or absent), severity (Clinical Global Impression of Severity Scale (CGI-S); range 1–7), and the “myoclonus at rest” part of the Unified Myoclonus Rating Scale (UMRS) score (range 0–128).11,12 Additionally, all stimuli were scored separately regarding stimulus sensitivity. If raters considered an item as not evaluable it was excluded from further analysis. The evaluations of PAM patients (total 3×10 = 30 evaluations) were ordered based on the total number of stimulus-sensitive stimuli (0, 1–2, 3–4, 5–6, 7–8, 9–10). These categories were corrected for group size, and correlated with how often a rater concluded “stimulus-sensitive PAM”. Finally, painful stimuli (pain stimulus of orbit, trapezius muscle, nail bed, and pin prick) were compared with non-painful stimuli in evoking PAM.
Inter-rater variability was calculated by the intraclass correlation coefficient (ICC) and kappa (Fleiss) statistics.13 The two-way mixed model and single measurement coefficients of ICCs were used. According to Landis and Koch,14 we interpreted kappa and ICC outcomes as follows: <0.00 = poor; 0.00–0.20 = slight; 0.21–0.40 = fair; 0.41–0.60 = moderate; 0.61–0.80 = substantial; >0.81 = almost perfect. The correlation between the amount of stimulus-sensitive stimuli and the conclusion “stimulus sensitive PAM” was performed using Spearman’s correlation. Stimulus sensitivity of PAM after painful and non-painful stimuli was compared using a two-sided Fisher’s exact test. p-Values of <0.05 were considered as statistically significant.
Patients had a mean age of 59 (SD±16) years, and seven were male (Detailed patient characteristics are provided in Table 1). All patients were treated with target temperature management. One patient had a good recovery with only mild cognitive deficits after 6 months (Case 2 in Table 2, Video 1). All other patients died in the acute setting. Patients were examined a mean of 9 hours (range 2–24 hours) after the first appearance of PAM. In seven patients no symptomatic treatment was given at the moment of examination.
Poor agreement was found among raters in assessing PAM phenotype (generalized or (multi)focal) (κ = –0.05, 95% confidence interval (CI) –0.40 to 0.31). Agreement for the localization of PAM (proximal, distal, or both) was moderate (κ = 0.46, 95% CI 0.20–0.73). In the rating scales used to assess PAM severity, substantial agreement was obtained for the CGI-S (ICC 0.64, 95% CI 0.14–0.89) and almost perfect agreement for the UMRS (ICC 0.82, 95% CI 0.57–0.95) (Figure 1). The specific scores of the raters are provided in Table 2.
In total 294 of 320 (92%) stimuli were examined by the raters. Poor agreement was found for confirming the presence or absence of “stimulus-sensitive PAM” in a patient (κ = –0.05, 95% CI –0.40 to 0.30) (Figure 1, Table 2). Likewise, when raters examined all 294 stimuli regarding the effect on PAM, fair agreement was found (κ = 0.276, 95% CI 0.21–0.34). A positive correlation was present between the conclusion “stimulus-sensitive PAM” and the amount of stimuli rated positive (r = 0.75, p<0.001). Finally, painful stimuli were significantly more likely (p<0.001) to evoke PAM than non-painful stimuli (respectively 21% and 7%).
Our results suggest that the clinical assessment of myoclonus in the first few days after anoxic brain injury is not consistent between different raters. Since phenotype, localization, and stimulus sensitivity findings of PAM varied considerably between raters the potential to accurately determine the etiology, i.e. cortical versus subcortical, was poor. We therefore conclude that clinical assessment of PAM should not be used for PAE prognostication or guiding treatment decisions. This is in line with the current European recommendations, which do not include PAM as a single outcome predictor for PAE.8
The clinical phenotype of PAM appears to be a continuum ranging from severe generalized PAM on one side of the spectrum and subtle focal PAM on the other side. Most likely, the PAM patients on one end of the spectrum can be assessed reliably, but as most cases show jerks in the middle of the spectrum clinical assessment is complicated and not interpreted consistently between raters.4,5 Other factors that are known to complicate the clinical evaluation of PAM are the dynamic character and cyclicality, sporadic nature, stimulus dependency, use of sedatives or neuromuscular blockers, and the presence of other involuntary movements.8,15 In our study, these factors could have caused the differences in interpretation, but we were unable to investigate this because of the small sample size. Future studies towards the etiology of PAM should use electroencephalography–electromyography to determine the origin of the myoclonus. Electrophysiology might prove a better tool to discriminate between cortical and subcortical myoclonus and a possible tool to use for prognosis.10
The CGI-S scale and especially the UMRS appear to represent reliable assessments of PAM severity. The relation between PAM severity and outcome is however unknown. The CGI-S and UMRS are recognized as reliable scales, but have not previously been used for PAM in particular.11,12 Moreover, in research evaluating the treatment of PAM, the UMRS could be reliably used to assess intervention effects.
An additional finding of our study was that painful stimuli induced significantly more PAM than non-painful stimuli. Since painful stimuli activate the ascending arousal system (AAS),16 subcortical structures could be triggered. Alternatively, since AAS promotes cortical excitability,16 (cortical) PAM might occur.
A potential limitation of our study was that assessments were based on video recordings leading to an observation that is less vivid. Contrarily, video recordings ensure that raters have the same information and the use of video is well established in movement disorders. Furthermore, the use of video enables the raters to assess myoclonus multiple times and to look at different parts of the video at different moments. Another limitation was the relatively small number of patients included.
In conclusion, our study shows that the clinical assessment of acute PAM is not consistent between physicians, and therefore the use of PAM for prognostication or guiding treatment decisions in PAE should be restrained. PAM severity measured by the UMRS appears to be reliable; however, the relation between PAM severity and outcome has not yet been established.
Supplement A referenced in this article is available here: https://doi.org/10.7916/D85T3XQK.
1. Wijdicks EF, Parisi JE, Sharbrough FW. Prognostic value of myoclonus status in comatose survivors of cardiac arrest. Ann Neurol 1994;35:239–243. doi: 10.1002/ana.410350219
2. Gupta HV, Caviness JN. Post-hypoxic myoclonus: current concepts, neurophysiology, and treatment. Tremor Other Hyperkinet Mov 2016;6:409. doi: 10.7916/D89C6XM4
3. Seder DB, Sunde K, Rubertsson S, Mooney M, Stammet P, Riker RR, et al. International Cardiac Arrest Registry: neurologic outcomes and postresuscitation care of patients with myoclonus following cardiac arrest. Crit Care Med 2015;43:965–972. doi: 10.1097/CCM.0000000000000880
4. Bouwes A, van Poppelen D, Koelman JH, Kuiper MA, Zandstra DF, Weinstein HC, et al. Acute posthypoxic myoclonus after cardiopulmonary resuscitation. BMC Neurol 2012;12:63. doi: 10.1186/1471-2377-12-63
5. van Zijl JC, Beudel M, vd Hoeven HJ, Lange F, Tijssen MAJ, Elting JWJ. Electroencephalographic findings in posthypoxic myoclonus. J Intensive Care Med 2016;31:270–275. doi: 10.1177/0885066615571533
6. Young GB, Gilbert JJ, Zochodne DW. The significance of myoclonic status epilepticus in postanoxic coma. Neurology 1990;40:1843–1848. doi: 10.1212/WNL.40.12.1843
7. Wijdicks EF, Hijdra A, Young GB, Bassetti CL, Wiebe S. Quality Standards Subcommittee of the American Academy of Neurology. Practice parameter: prediction of outcome in comatose survivors after cardiopulmonary resuscitation (an evidence-based review): report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology 2006;67:203–210. doi: 10.1212/01.wnl.0000227183.21314.cd
8. Sandroni C, Cariou A, Cavallaro F, Cronberg T, Friberg H, Hoedemaekers C, et al. Prognostication in comatose survivors of cardiac arrest: an advisory statement from the European Resuscitation Council and the European Society of Intensive Care Medicine. Resuscitation 2014;85:1779–1789. doi: 10.1016/j.resuscitation.2014.08.011
9. Rossetti AO, Rabinstein AA, Oddo M. Neurological prognostication of outcome in patients in coma after cardiac arrest. Lancet Neurol 2016;15:597–609. doi: 10.1016/S1474-4422(16)00015-6
10. Cassim F, Houdayer E. Neurophysiology of myoclonus. Neurophysiol Clin 2006;36:281–291. doi: 10.1016/j.neucli.2006.10.001
11. Busner J, Targum SD. The Clinical Global Impressions Scale: applying a research tool in clinical practice. Psychiatry (Edgmont) 2007;4:28–37.
12. Frucht SJ, Leurgans SE, Hallett M, Fahn S. The Unified Myoclonus Rating Scale. Adv Neurol 2002;89:361–376.
13. Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol 2011;64:96–106. doi: 10.1016/j.jclinepi.2010.03.002
14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159. doi: 10.2307/2529310
15. Samaniego EA, Mlynash M, Caulfield AF, Eyngorn I, Wijman CAC. Sedation confounds outcome prediction in cardiac arrest survivors treated with hypothermia. Neurocrit Care 2011;15:113–119. doi: 10.1007/s12028-010-9412-8
16. Schwartz JR, Roth T. Neurophysiology of sleep and wakefulness: basic science and clinical implications. Curr Neuropharmacol 2008;6:367–378. doi: 10.2174/157015908787386050
Table 1. Clinical Characteristics
|Abbreviations: CPR, Cardiopulmonary Resuscitation; EEG, Electroencephalography; ICU, Intensive Care Unit; IHCA, In Hospital Cardiac Arrest; IQR, Interquartile Range; OHCA, Out of Hospital Cardiac Arrest; PAM, Post-anoxic Myoclonus; PEA, Pulseless Electrical Activity; ROSC, Return of Spontaneous Circulation; SD, Standard Deviation; SEP, Somatosensory Evoked Potential; VF, Ventricular Fibrillation; VT, Ventricular Tachycardia.|
|Age, years, mean (SD)||59||(16)|
|Initial rhythm, n|
|Primary cause of CPR, n (%)|
|Location of arrest, n|
|Time to ROSC (minutes), median (IQR)||18||(10–23)|
|Time to occurrence of PAM (hours), median (IQR)||14||(10–36)|
|Initial treatment of PAM, n|
|SEP N20, n|
|EEG result, n|
|Recovery with mild cerebral disability||1|
|Treatment withdrawal, n||9|
|Time to treatment withdrawal (hours), median (IQR)||48||(38–130)|
|Reason for treatment withdrawal, n|
Table 2. Video Assessment Scores of Post-anoxic Myoclonus
|PAM Phenotype||Stimulus Sensitivity||Localization||CGI–S||UMRS|
|Abbreviations: +, stimulus sensitivity present; –, stimulus sensitivity absent; CGI-S, Clinical Global Impression of Severity Scale; d, Distal; gz, Generalized; mf, (Multi)focal; p, Proximal; PAM, Post-anoxic Myoclonus; R, Rater; UMRS, Unified Myoclonus Rating Scale.|
Figure 1. Inter-rater Variability of Acute Post-anoxic Myoclonus. Inter-rater variability scores with 95% confidence limits of the different clinical items of post-hypoxic myoclonus assessed by three experienced neurologists.
Video 1. Example Examination of a Post-anoxic Myoclonus Patient. The (abridged) systematic examination of post-anoxic myoclonus (PAM) Case 2 (Table 2) and the video protocol of this study (Supplement A). Case 2 displays slight myoclonus in rest, but it seems to increase in frequency and severity after the application of stimuli. This patient is the only PAM case of this cohort that survived and recovered with only mild cognitive deficits after 6 months.