In our industry, we religiously reference clinical studies to promote our products. While this is very important to establish credibility do you know that not all clinical papers are equal? At best they are a reliable reference that could influence medical practices and at worst a source of disinformation that may ultimately impact negatively on patients wellbeing.
Thus, there is a need to ensure that the clinical papers we use are from a reputable source and conform to certain basic principles. Is the article published in a well-known journal or is it something published in a tabloid magazine? Usually, articles published in a respected journal have been peer-reviewed to ensure credibility. Something else that is important is the date of publication – something published 30 years ago is probably not relevant today.
Look for the level of evidence – this is rated from 1-5. Level one usually refers to a prospective (the trial was structured towards looking at specific endpoints), randomized subjects (to reduce potential bias) often with a comparison to a chosen baseline. There must also be enough subjects to reduce the impact of chance or coincidence affecting the results. This is not always easy, especially in medical device studies.
A level 2 study often refers to a smaller randomized often with less clear results. These may also be retrospective with different conclusions drawn at a later stage from what the study was initially designed for. Level 3 refers more to carefully controlled case studies or even a discussion published from looking at many different, previously published papers.
Level 4 will often incorporate a series of case studies but will not involve sufficient patient numbers, comparison controls, proper statistical analysis and is generally not taken very seriously by the scientific community.
Level 5 generally refers to ‘expert’ opinion (who designates them as experts?) or a single case study. Editorial comments and published letters may fall into this category. Many companies promote this as gospel!
A handy tool to refer to in the results is the p-value. This is a statistical calculation to reflect the validity of the results (simply speaking will your data reflect similar results if conducted again under similar conditions?). Usually, if the p-value is less than 0.05 then your data is said to be statistically significant and more reliable and less influenced by chance. Studies with a p-value of less than 0,001 are said to have highly valid results.
Look closely who is sponsoring the study and the authors. Level 4 data sponsored by a company may warrant closer inspection of the data! Not all industry-sponsored papers are to be discredited as the industry does play a valuable role in scientific research.
So go grab some clinical papers and look at them with a fresh, critical eye! Who knows what you might find!