Pain is the most unpleasant experience experienced by critical care patients because it can disturb the patient’s physiology and psychology [1]. This pain is felt both when the patient is resting and during routine clinical procedures [2]. Some interventions that cause pain are the surgical process, endotracheal tube (ETT) installation, chest tube installation, hemodialysis (HD) catheterization, deep breathing and coughing exercises, endotracheal aspiration, wound care, patient mobilization and prolonged bed rest [3]. Pain in critical patients or decreased consciousness that is not resolved properly will cause tachycardia, decreased immune response, increased catecholamine production, and increased oxygen consumption. Patients who experience severe pain can develop acidosis, the occurrence of which is often associated with increased pain scores. Pain felt when the patient is resting will also cause multisystemic complications which will have a serious impact on the patient’s quality of life and well-being after treatment [4].
Pain scale assessment in critical care patients only used in 1.6% of 183 cases. Although behavioral evaluation of pain is generally reported in 73% of cases, these evaluations and observations are carried out without valid and reliable pain assessment tools. Research conducted on 3601 unconscious patients found that 53% of patients who used painkillers did not have their pain scale assessed. Meanwhile, only 28% of pain evaluations were carried out using appropriate and specific tools [5].
Regular pain assessments using a valid pain assessment tool will help nurses document pain assessments well. Well-controlled pain levels will lead to the achievement of desired goals, including reduced hospitalizations, decreased need for painkillers and sedatives, reduced nosocomial infections and deaths, improved quality of life, and increased patient satisfaction [6].
The assessment tool that can be used must be valid and reliable and appropriate to the development and culture of the target population, available in multiple languages or can be easily translated, easy for others to understand or understand, health teams like to use the assessment, easy to obtain, reproduce and distribute [7]. One tool that has been developed to assess pain in critical patients or decreased consciousness being treated in the ICU is the Critical Care Pain Observation Tool (CPOT). The Critical-Care Pain Observation Tool (CPOT) developed by Gelinas et al [8], has four assessment categories, namely, facial expressions, body movements, muscle tension and compliance with the ventilator for intubated patients or vocalizations for non-intubated patients. CPOT was originally developed in French and has been translated into several languages. Based on the results of its strong psychometric analysis, CPOT is a recommended instrument for unconscious or critical patients [9]. This literature review aims to identify and summarize the results of validity and reliability tests of the CPOT instrument from various languages.
Cultural adaptation is a process that examines language translation and cultural issues in the process of preparing instruments to be used elsewhere [10]. According to Guillemin et al (1993), the word culture used in cultural/cross-cultural adaptation is a combination of literal translation of individual words and sentences from one language to another and adaptation by referring to idioms and cultural contexts and lifestyles. Therefore, this study examines the cultural adaptation process of selected articles through the following variables: (a) translation (linguistic validation) and (b) evaluation of the psychometric properties of an instrument. CPOT has been adapted and validated for use in various languages and cultural contexts. Most validation studies evaluated the psychometric properties of CPOT in intensive care patients who are unable to communicate verbally such as critically ill patients or patients with impaired consciousness [12].
In this review, the research methodology, translation procedures and psychometric measures of CPOT versions from various languages will be identified. By collecting information about the validity and reliability of CPOT across languages, a better comparison of the quality of pain assessment results in critically ill patients or patients with decreased consciousness will be obtained.
Research question
The research questions for this scoping review are as follows:
1. How many versions of CPOT have been validated?
2. What is the methodology for validating each version of CPOT?
Method
The primary aim of this scoping review is to identify and synthesize all published CPOT validation studies across various databases. The articles in this review follow a framework based on Arksey and O’Malley (2005) and a new framework developed by Levac et al (2010). This review consisted of five stages: (a) identifying research questions, (b) identifying relevant ones, (c) study selection, (d) data extraction, and (e) Compiling, summarizing and reporting the results.
Scoping reviews are a useful tool in improving the quality of evidence synthesis approaches [15]. At different times, these reviews are conducted when the purpose of the review is to identify knowledge gaps, cover a body of literature, clarify concepts, investigate the conduct of research, or inform a systematic review. Different from other literature observations such as systematic observations or meta-analysis, this scoping observation aims to map relevant CPOT validation studies.
Search Strategy
Data were obtained from searching articles in major databases such as PubMed, Science Direct and Google Scholar. The search was carried out using the keyword «Critical Care Pain Observation Tool». Because CPOT is a specific assessment of Pain Assessment, we did not look for other types of assessment. Double quotes (» ») are used to search for more specific phrases. without using double quotation marks, many irrelevant articles were identified. The “snowball” technique is also applied in article searches, where references from selected articles are also checked to identify articles that may not be found in database searches. To store, manage publication data and remove duplicates, applications or reference devices are used.
Study Selection
The identified articles were carefully checked. Articles were selected based on inclusion criteria, namely: (a) the article is an original study, (b) the article describes the process of adapting or validating the CPOT and (c) the article is a peer review. Exclusion criteria are: (a) articles are conference abstracts, (b) non-English, (c) articles that discuss CPOT but do not validate CPOT.
This review aims to map methods and procedures for cultural adaptation and validation of CPOT. Thus, data extraction prioritized psychometric properties (reliability, validity and study group) and translation methods of CPOT validation studies. SS and RA are also responsible for the title and abstract selection process to assess the suitability of the article. Then full-text screening was performed on the remaining articles according to the inclusion and exclusion criteria. Then full-text screening was carried out on the remaining articles according to the inclusion and exclusion criteria. Data extraction is completed by collecting data, namely (a) publication information (publication date, author, article title, country of origin), (b) sample characteristics (population, sample size, age of participants), (c) psychometric properties (reliability, validity and boundary values), and (d) translation method (translation procedures and translation guidelines followed). After the article screening is complete, the results are assessed to discuss differences of opinion so that a consensus is reached.
Results
Search result
A flow diagram illustrating the literature search is shown in figure. A systematic search of the PubMed, Science Direct and Google scholar databases yielded a total of 1,086 articles. Then duplicate articles were excluded. After reading the titles and abstracts, 91 articles were deemed eligible for full-text screening based on exclusion criteria. The result was 13 research articles that met the eligibility criteria and were included in this observation. Validation studies came from 13 countries including France-Canada, Spain, Taiwan, Germany, Brazil, Turkey, Portugal, Poland, Persia, China, Italy, Norway and the Netherlands. All of these studies are included in the reflection.
Flowchart of Critical Care Pain Observation Tool Research Identification.
Блок-схема исследования.
Population
All selected studies validated CPOT in patients admitted to intensive care units, where patients were unable to report pain verbally (Table 1). There are 5 studies validating CPOT in patients who have undergone cardiac surgery, 4 studies validating CPOT in mechanical ventilation patients, 2 studies in intubated patients and 1 study in brain injury patients. There was 1 study that was validated in a mixed group of patients, namely critically ill patients, patients with decreased consciousness, patients after surgery and patients on mechanical ventilation.
Таблица 1. Этиология, значимость и пороговые значения выбранных исследований
| CPOT Version | Population | Interrater Reliability | Intern Consistency (Cronbach Alpha) | Test Retest (Spearman rho) | Intraclass Correlation Coefficient | Cutoff Values (Sensitivity/Specificity; area under the curve? |
| France-Canada (Gelinas et al, 2021) | Brain Injury | 0.76 | — | 0.4—0.6 | 0.69 | 2 (77%/69%; 0.76) |
| Spannish (Vazquez et al, 2011) | Mechanically ventilated patients | 0.79—1 | — | — | — | — |
| Traditional Chinese (Hsiung et al, 2016) | Surgical ICU Patients | — | 0.697 | — | — | — |
| German (Kiesewetter et al, 2019) | Cardiac Surgery ICU Patient | 0.73 | 0.8 | — | — | — |
| Brazilian Portuguese (Klein et al, 2018) | Surgical ICU Patients | 0.7 | — | — | — | — |
| Türkiye (Aktas et al, 2017) | Cardiac Surgery ICU Patient | 0.55—1.00 | 0.72 | — | — | — |
| Portugis Eropa (Marques et al, 2022) | Mechanically ventilated patients | 0.93—0.99 | — | — | — | 5 (71%/80%; 0.764) |
| Polish (Kotfis et al, 2017) | The patient is intubated | 0.97 | — | — | 0.85 | — |
| Persian (Zand et al, 2015) | Surgical Patients | — | 0.86 | — | 0.39—0.85 | — |
| Chin (Li et al, 2014) | Mechanically ventilated patients | 0.80—0.91 | 0.57—0.86 | 0.81—0.93 | — | 2(80.8—89.4%/73.3—81.8%; 0.849—0.902) |
| Itali (Sulla et al, 2017) | ICU patient | 0.78—0.86 | — | 0.42—0.99 | — | — |
| Norwegian (Storsveen et al, 2017) | Mechanically ventilated patients | 0.41—0.82 | — | — | 0.68—0.80 | — |
| Dutch (Stilma et al, 2015) | The patient is intubated | 0.38—0.56 | 0.56 | — | — | 2 (39%/85%; 0.65) |
Inter-rater Reliability
Inter-rater reliability is the consistency of measurements or observations agreed upon by two raters. Inter-rater reliability measurements were assessed using the kappa coefficient. Observations were made before, during and after the nociceptive procedures were carried out. Kappa values of 0.81—1.0 indicate excellent fit, 0.61—0.80 good, 0.41—0.60 moderate, 0.21—0.40 poor, and <0.21 poor [36]. In this review, there were 2 studies that did not assess inter-rater reliability, namely Traditional Chinese and Persian.
Internal Consistency
Internal consistency is one measure used to assess reliability in all selected studies. A summary of the reliability of all the studies has been summarized in Table 1. As indicated by the Cronbach’s Alpha coefficient, the internal consistency of the studies included in this review varied from 0.57—0.86. Only 6 studies assessed Internal Consistency, namely, Hsiung et al (2016), Kiesewetter et al (2019), Klein et al (2018), Zand et al (2015), Li et al (2014) and Stilma et al (2019).
Test-Retest Reliability
Test-Retest reliability is a commonly used measure to assess reliability. This assessment is determined using the Spearman correlation coefficient to evaluate the results of assessments that have been carried out over time. Only 3 validation studies assessed test retest reliability, namely Gelinas et al (2021), Li et al (2014) and Sulla et al (2017). The values vary from 0.4 to 0.99 (See table 1). In this review the time range for measurements generally varied from 15—20 minutes after the procedure.
Cut Off Score
Three of the 13 studies included a cutoff score for CPOT. Studies presenting CPOT cut-off values show quite high diagnostic sensitivity (71—89.4%) and specificity (69—83%). In previous studies with the original version of the CPOT, the cut-off score varied from 2 to 3 to define the presence of pain in the ICU patient population. Determining the cut-off value for moderate pain to severe pain will be more widely used in the pain management decision making process to determine the time of use of opioids that is most needed.
Validity
A summary of the validity of the selected studies is in Table 2. Almost all validation studies measure discriminative validity, except for 2 studies, namely Hsiung et al, 2016 and Stilma et al, 2015. Discriminant validity for CPOT was analyzed by differences in CPOT scores at rest, during the nociceptive procedure and 15—20 minutes after the procedure. Assessments were measured in all domains: facial expressions, body movements, compliance with the ventilator, vocalizations and muscle tension. Criterion validity was assessed by categorizing patients with and without pain and different levels of pain intensity during nociceptive procedures. There were 5 studies that did not assess the validity of the criteria, namely Spanish, Traditional Chinese, Brazilian Portuguese, Turkish and Norwegian.
Таблица 2. Параметры выборок и достоверность исследований
| Versi CPOT | Patient | Construct Validity | Concurrent Validity | Content Validity | Convergen Validity | Criterion Validity | Discriminatif Validity | Predictive Validity | |
| n | M usia | ||||||||
| France-Canada (Gelinas et al, 2021) | 226 | 53—58 | — | — | — | — | + | + | — |
| Spannish (Vazquez et al, 2011) | 96 | 22—89 | — | — | — | — | — | + | — |
| Traditional Chinese (Hsiung et al, 2016) | 10 | 40—84 | — | — | + | — | — | — | — |
| German (Kiesewetter et al, 2019) | 292 | 68,5 | — | — | — | — | + | + | — |
| Brazilian Portuguese (Klein et al, 2018) | 124 | 65 | — | — | — | — | — | + | + |
| Türkiye (Aktas et al, 2017) | 66 | 44—82 | — | + | + | — | — | + | — |
| Portugis Eropa (Marques et al, 2022) | 110 | 20—95 | — | — | — | + | + | + | — |
| Polish (Kotfis et al, 2017) | 71 | 66 | — | — | — | — | + | + | — |
| Persian (Zand et al, 2015) | 66 | 50 | — | — | — | — | + | + | — |
| Chin (Li et al, 2014) | 63 | 61 | — | — | — | — | + | + | — |
| Itali (Sulla et al, 2017) | 50 | 42 | — | — | — | — | + | + | — |
| Norwegian (Storsveen et al, 2017) | 18 | 31—84 | — | — | — | — | — | + | — |
| Dutch (Stilma et al, 2015) | 108 | 24—91 | — | — | — | — | + | — | — |
CPOT Translation
Translation is a general stage in translating and validating a questionnaire in a new language. Even though CPOT’s original language is French, CPOT developers also include an English translation of CPOT in their publications. Table 3 shows a comparison of the CPOT translation stages in various languages. All validation studies used stages of forward and backward translation, which are known methods for achieving similarity between two languages in cross-cultural research [10]. Based on the method proposed by Brislin 1970, a bilingual translator is a translator who is competent to translate instruments from the original language to the target language (forward translation). Then, another bilingual translator will translate the instrument back into the original language (backward translation). The two translation results must be compared for their suitability. There are 4 translation procedures documented in the research, namely ISOPRO (Wild et al, 2005); Harkness 2003, Streiner and Norman, 2008; Beaton et al, 2000) and the World Health Organization (WHO).
Таблица 3. Описание процесса адаптации шкалы
| Language | Translated by more than 3 people | Translated by lay people | People involved in translation | Back Translation | Translation guidelines |
| France-Canada (Gelinas et al, 2021) | — | — | — | + | — |
| Spannish (Vazquez et al, 2011) | — | — | — | — | — |
| Traditional Chinese (Hsiung et al, 2016) | + | — | Forward Translation: 2 translators with master’s degrees in English. Back Translation: 2 translators, a clinical doctor | + | Harkness, 2003; Streiner & Norman, 2008) |
| German (Kiesewetter et al, 2019) | — | — | — | + | ISPRO (Wild et al, 2005) |
| Brazilian Portuguese (Klein et al, 2018) | + | — | Forward Translation: 1 translator has a linguistic background and English is a second language. 1 translator with Health knowledge. Back Translation: 2 original translators: English with Brazilian Portuguese is the second language | + | Beaton et al, 2000 |
| Türkiye (Aktas et al, 2017) | + | — | Consisting of 3 specialist Turkish translators who work independently as translators | + | — |
| Portugis Eropa (Marques et al, 2022) | + | — | Forward Translation: 2 professional translators. Back Translation: 2 professional translators in English | + | Beaton et al, 2000; Reichenheim, 2007 |
| Polish (Kotfis et al, 2017) | — | — | English translation team from Vanderbilt University | + | — |
| Persian (Zand et al, 2015) | + | — | Forward Translation: 3 ICU nurses who translate using an English to Persian dictionary. Back Translation: 1 doctor and 1 ICU nurse who are fluent in English and do not know about CPOT | + | — |
| Chin (Li et al, 2014) | — | — | — | + | WHO |
| Itali (Sulla et al, 2017) | — | — | — | + | — |
| Norwegian (Storsveen et al, 2017) | + | — | Forward Translation: 2 ICU nurses fluent in Norwegian and English, 1 researcher who independently translated CPOT. Back Translation: 1 professional translator | + | ISPRO (Wild et al, 2005) |
| Dutch (Stilma et al, 2015) | + | — | Forward Translation: 2 professional translators, one of whom is an ICU nurse. Back Translation: 1 professional translator who does not know the original CPOT | + | COSMIN Cecklist (Terwee et al, 2012) |
Discussion
This research is research that discusses cultural adaptation and the CPOT translation process. In selected research articles, researchers not only identify the translation process but also the reliability and validity of articles that meet the criteria. The articles in this review, stratify the populations included in CPOT validation studies. All populations in this study were patients treated in the ICU. It is known that the majority of patients treated in the ICU are patients who are unable to report the pain they feel verbally. This is caused by the use of sedation drugs, critically ill patients, patients with decreased consciousness and patients on mechanical ventilation.
Research that meets the inclusion criteria assesses reliability using internal consistency (Cronbach’s alpha), test-retest reliability (Spearman’s rho) and Inter-rater reliability (Coefficient Kappa). The Cronbach Alpha value ranges from 0.0 to 1.0 which shows how well the items measure the same basic dimensions [34]. In general, a Cronbach Alpha coefficient greater than 0.90 is acceptable as the minimum value for measurement [35]. The five studies in this review displayed Cronbach Alpha Coefficients between 0.57—0.86 indicating that the CPOT validated in various languages has good internal consistency.
Test-retest reliability is also known as reproducibility [36], which relates to the extent to which repeated assessments of individuals will produce similar answers. Test-retest reliability of the selected studies varied between 0.4 to 0.99 indicating similarity of results over time. The test—retest reliability of the selected studies varied between 0.4 to 0.99 indicating similarity of results over time.
The types of validity that are important in this review are discriminative validity, criterion validity, content validity [34]. Criterion validity shows the extent to which assessment items demonstrate the responses produced. Discriminative validity here measures pain scores in the resting state and clinical or nociceptive procedures are performed.
The current review identifies the number of translators and the use of translation techniques in the CPOT validation process. Translations have higher quality when carried out by at least two independent translators to help detect errors and interpretations that differ from the original version [11]. Backward translation, which translates back from the target language to the original language will improve the quality of the final version of the translation process [10]. Summarized in Table 3, CPOT from various languages uses backward translation. Based on the number of translators, only the traditional Chinese, Brazilian Portuguese, Turkish, European Portuguese, Persian, Norwegian and Dutch versions of CPOT clearly state the number of translators who participated in the translation process.
Conclusion
The findings of this review include the number of CPOT validation studies and their validation methods. These findings should provide information for future researchers in validating CPOT regarding the psychometric parameters of existing versions of CPOT and how to apply translation methods to adapt CPOT to the desired language and culture.
The authors have declared that no competing interests existed at the time of publication.