Published in Br J Anaesth. 2014;113(3):340-343.
It is fascinating and alluring—the evidence is converging that with goal-directed haemodynamic therapy we can actively contribute to better postoperative outcomes of our patients, particularly in our high-risk patients. And this is potentially possible, as suggested by an overwhelming number of publications from recent years, with completely non-invasive advanced haemodynamic monitoring. Indeed, it consequently marks the next step of a paradigm shift in haemodynamic monitoring that we have witnessed over the past two decades. While the pulmonary artery catheter (PAC) was widely used as the ‘holy grail’ in haemodynamic monitoring after its introduction in the 1970s up until the late 1990s, several large randomized controlled trials failed to demonstrate improved outcome in patients monitored with the PAC which led to a marked decline in its use over the following years. Although the PAC can still provide important information in patients with pulmonary arterial hypertension and right ventricular failure, there is increasing consensus that the PAC should not be routinely used as the primary means of advanced monitoring.
However, concerns over the safety of the PAC smoothed the way for the development of less-invasive monitoring technologies. These less-invasive technologies have included oesophageal Doppler, transpulmonary thermodilution, lithium indicator dilution and also calibrated and un-calibrated pulse contour analysis. Now, as the next logical step, manufacturers have introduced totally non-invasive devices for advanced haemodynamic monitoring, including thoracic bioimpedance, bioreactance, radial artery applanation tonometry, and the volume clamp method based on the Penaz principle. In their study published in a past issue of the British Journal of Anaesthesia, Vos and colleagues demonstrated that continuous non-invasive measurement of mean arterial pressure using the volume clamp method is not inferior compared with intermittent oscillometric measurements when using invasively assessed mean arterial pressure as the reference value.
But which of those new, or re-vitalized technologies really enable us to transfer the benefits of advanced haemodynamic management into our patients, that is, which of these devices can accurately measure changes in arterial pressure and flow under physiological and pathophysiological circumstances with the necessary degree of accuracy and precision, on which we can reliably base our therapeutic decisions?
Therefore, we are now confronted with a plethora of validation studies that aim to demonstrate that these devices provide measurements with clinically acceptable agreement compared with invasive devices, usually used as the criterion standard.
In general, readers of validation studies—because of shortcomings in study design, statistical analysis, and data interpretation—are unfortunately often left unsure of the definitive conclusions of the study results.
It is worth taking a step back to consider a few basic problems that one is faced with when evaluating such novel technologies. What are the variables that are available non-invasively and suitable for guiding therapy in a perioperative or intensive care unit setting? What is the appropriate criterion standard against which new devices should be tested? What are the appropriate statistical methods for the comparison with an alleged criterion standard technique? Regarding the agreement with a criterion standard method, which performance requirements need to be met by non-invasive monitoring technologies? Is appropriate trending more important than absolute accuracy and precision? What practical requirements need to be met by non-invasive monitoring devices regarding their use in daily clinical practice? Which patients can benefit from continuous non-invasive monitoring? What goal-directed therapy protocols can be applied with non-invasive technologies?
While it is far beyond the scope of this editorial to discuss the value of goal-directed therapy approaches, it can be considered generally accepted that treatment aimed at optimizing mean arterial pressure and global blood flow in terms of cardiac output can improve patient outcome in the perioperative setting. Therefore, a completely non-invasive technology capable of providing arterial pressure, cardiac output, and functional cardiac preload parameters (i.e. stroke volume variation or pulse pressure variation) accurately and in a continuous manner is desired. In principle, devices using the volume clamp method or radial artery applanation tonometry can fulfil these requirements. In principle yes, but under which circumstances—this is the question to be answered. It is not as important to know that the fuel tank display in my car works accurately and is precise when the fuel tank is half full, but the question becomes relevant if I have to make the decision as to whether or not to leave the motorway at the next filling station, even if this is the most expensive in the area, or if there is enough fuel to get to a cheaper station further away, when the display is blinking ’empty’.
Furthermore, when performing ‘validation’ studies that compare non-invasive monitoring technologies with routinely used ‘gold standard’ techniques, the question as to which technique should be used as the criterion standard needs to be carefully considered. While invasive assessment with an arterial catheter is considered the criterion standard for arterial pressure determination, the experimental criterion standard for cardiac output measurement is direct blood flow measurement, and for clinical studies, it is thermodilution. However, are these invasively assessed parameters really the standard that needs to be met by a completely non-invasive device? Or should we rather temper our expectations regarding the measurement performance of these new non-invasive systems? Should a non-invasive device rather be tested against a clinically established less-invasive technology or even another completely non-invasive technology in pragmatic studies that are aimed at proving ‘non-inferiority’ of the new devices? Instead of claiming that a non-invasive device must accurately match arterial pressure and cardiac output values obtained with an arterial catheter or a PAC, respectively, perhaps we should rather try to demonstrate that these devices are as accurate as arterial pressure values obtained by oscillometry or cardiac output values assessed by non-calibrated pulse contour analysis.
Sound statistical methods for the comparison between a criterion standard method and a non-invasive technique are crucial for the proper assessment of a device’s accuracy and precision. Bland–Altman analysis accounting for the fact that repeated measurements per individual are analysed and the calculation of the percentage error have become the basic statistical analyses in method comparison studies in the field of haemodynamic monitoring.
When evaluating non-invasive monitoring technologies in clinical studies, the fundamental problem arises that we generally lack accepted definitions for ‘clinically acceptable agreement’ or ‘interchangeability’. The fact that predefined performance requirements regarding accuracy and precision of non-invasive devices are still missing has led to a variety of misinterpretations of data and scientifically incorrect conclusions. For instance, some studies on continuous non-invasive arterial pressure monitoring report their results citing the American National Standards Institute/Association for the Advancement of Medical Instrumentation (ANSI/AAMI) standards. However, this ANSI/AAMI ‘standard’ was developed to evaluate non-automated, automated, or electronic sphygmomanometers ‘that are used with an occluding cuff for the indirect determination of arterial blood pressure’ and can therefore not be transferred to non-invasive technologies that continuously assess arterial pressure such as the volume clamp method or radial artery applanation tonometry.
In studies on non-invasive cardiac output determination, most authors refer to the percentage error by Critchley and Critchley and its ‘magic cut-off value’ of 30% (that should in fact be 28.3%). However, most authors ignore the fact that this percentage error depends on both the precision of the reference method and the precision of the investigated method. In this context, the major prerequisite for the correct application of the 30% cut-off value is a precision of ±20% for both methods—a prerequisite that is hard to test and to fulfil with a variety of established cardiac output monitoring systems such as pulse contour analysis. Needless to say the percentage error cut-off value of 30% (defined to compare cardiac output measurements) cannot simply be applied to compare the measurement agreement between devices assessing other haemodynamic variables such as arterial pressure. When we consider the rapidly increasing number of studies that claim to validate non-invasive technologies, we urgently need to define generally applicable and accepted standards defining ‘clinically acceptable agreement’ between non-invasive and different established invasive haemodynamic monitoring technologies. Although these statistical pitfalls have previously been addressed, they are repeatedly ignored or misunderstood in method comparison studies that present accuracy and precision of new non-invasive technologies.
To further complicate matters, it has repeatedly been asked whether appropriate trending of haemodynamic variables might be more important than absolute accuracy and precision. The accurate detection of a decrease or increase in certain haemodynamic variables in response to diagnostic or therapeutic interventions (passive leg raising, fluid challenge, and administration of vasoactive agents) might in fact be very valuable in the treatment of critically ill patients. To assess the trending ability of different technologies established statistical analyses such as the four-quadrant plot as well as recently developed and more sophisticated approaches including the polar plot analysis are available. Clinical studies should therefore not only focus on accuracy and precision analysis under static conditions in haemodynamically stable patients, but should also include data on the ability of a non-invasive technique to rapidly follow changes in haemodynamic variables.
To be applicable in daily clinical practice, devices used for non-invasive haemodynamic monitoring should additionally be easy to set up, work operator-independent, and be resistant to artefacts caused by movement, peripheral oedema, or changes in vasomotor tone (e.g. in septic patients treated with vasoactive agents). However, all available technologies still have system-related weaknesses under one or more of the above-mentioned conditions. To overcome these shortcomings research on non-invasive haemodynamic monitoring technologies must therefore also continue in order to improve the clinical applicability of the new devices.
It must be borne in mind that clinical research is performed to improve patient outcome and not solely to compare one device with another. Therefore, we finally need to ask which patients in general can benefit from continuous non-invasive monitoring. It is agreed that despite the development of non-invasive advanced haemodynamic monitoring technologies patients treated in an intensive care unit for multiple organ dysfunction syndrome or high-risk surgical patients undergoing major surgery will, for the foreseeable future still be equipped with an arterial and central venous (or even pulmonary artery) catheter. However, there is a variety of clinical settings in which continuous non-invasive monitoring of arterial pressure and cardiac output might improve patient safety and outcome compared with clinical standard care (e.g. using intermittent oscillometric arterial pressure measurements). This might for instance hold true for patients undergoing intermediate-risk surgery, complex endoscopic procedures, interventional radiology procedures, or for patients treated in high-dependency units, stroke units, or emergency departments.
Most important, it needs to be re-emphasized that no haemodynamic monitoring technology per se (whether invasive or non-invasive) will ever change patient outcome unless coupled with a therapeutic medical intervention able to improve outcome. Therefore, following on from methodologically sound validation studies of new non-invasive haemodynamic monitoring technologies, studies that investigate whether goal-directed therapy algorithms based on parameters assessed with these technologies improve patient outcome need to be undertaken.
To conclude, today several advanced haemodynamic variables can be assessed non-invasively by a variety of different technologies. To be able to draw appropriate conclusions from validation studies that evaluate these technologies, appropriate criterion standard methods need to be defined and appropriate statistical methods for the assessment of accuracy, precision, and trending abilities need to be applied. Definitions for ‘interchangeability’ or ‘clinically acceptable agreement’ of non-invasive technologies and established invasive reference methods still need to be defined. To investigate the potential beneficial impact of continuous non-invasive monitoring, goal-directed approaches based on non-invasively assessed advanced haemodynamic variables must be evaluated.
We would like to leave the reader with the following thought: in our opinion, in anaesthesiology, we have already lived in the era of non-invasive monitoring for many years. On the one hand, anaesthesiologists all over the world base relevant clinical decisions on non-invasive monitoring technologies such as intermittent oscillometric arterial pressure measurements and pulse oximetry despite data that clearly demonstrate the inaccuracy of these methods in certain clinical situations. On the other hand, there is a convincing body of evidence that goal-directed optimization of advanced haemodynamic variables reflecting global blood flow improves patient outcome. Combining these thoughts, we should therefore ask ‘Are we ready for the age of advanced non-invasive haemodynamic monitoring?’ We have the alluring vision that in the near future it will be possible to improve patient outcome by goal-directed optimization of non-invasively assessed haemodynamic parameters. To achieve this goal, we must now perform clinical studies on advanced non-invasive haemodynamic monitoring meticulously and in a responsible manner, in order to avoid confusion resulting from incorrect conclusions based on inappropriately applied statistical methods.