Eye on EBVM: evaluating veterinary information

In the previous “Eye on EBVM” (VNT17.06), we considered how to find veterinary information. This time, we look at the next step – how to evaluate it.

What is critical appraisal?

Critical appraisal (CA) is the objective assessment of the quality of a piece of literature, considering:

• study design
• methodology
• results
• validity

By appraising a piece of veterinary literature critically, it is possible to appreciate its clinical importance and whether such information can be applied in a practice context.

As a veterinary nurse, the ability to assess, summarise and apply information from veterinary literature to everyday clinical scenarios is a valuable skill to develop – it also enables evidence-based nursing practice. CA skills are developed over time, however, and get easier with practice.

Before even starting your CA, try to familiarise yourself with the basic structure of the paper and the type of study design, as different types can influence the level of evidence the study provides (Buckley, 2017).

Also, seek out RCVS Knowledge Evidence-based Veterinary Medicine (EBVM) Toolkits 3 to 11 for advice on how to identify the type of study, as well as which checklist to use for different studies.

CA in six steps

Modified from Barker and Kane (2016), and Buckley (2017):

1. Find a quiet area where you will not be interrupted.
2. Read the introduction to see if it is relevant to the clinical question or scenario you are interested in.
3. Look at the “methods” section to see if the study design is appropriate for the type of question it is addressing (RCVS Knowledge EBVM Toolkits 3 and 4 have more information).
4. If the paper addresses your clinical situation and the study design is appropriate for the research question, appraise the quality of the study using a checklist (see RCVS Knowledge EBVM Toolkits 6 to 11).
5. Focus on the study aims (are they clearly defined?), the methodology (are they appropriate?) and results (consider their quality and relevance to a clinical scenario).
6. Identify the type of study, complete the appropriate checklist and discuss findings.

Assessing study quality

Now we’ve looked at what the study set out to do, and how it reached its conclusion, it is now time to ask: is it actually any good?

Is the study valid?

The first step is to decide whether the study was unbiased by evaluating its methodological quality:

• Is the title representative of the report?
• Who paid for it?
• Was ethical approval obtained?
• Who are the authors who funded the study?
• Are the aims clearly stated?

What are the results?

If we decide the study is valid, we can consider the results:

• Was the sample size justified?
• Are the measurements likely to be valid and reliable?
• Are the statistical methods described?
• Do the numbers add up? Are all the subjects in the study accounted for?
• Was the statistical significance assessed?
• What do the main findings mean?

Are the results useful?

Once you have decided the study and results are valid, you need to consider how this information applies to your question.

What CA can do

Social media and the internet abounds with animal health information, with our clients readily seeking Dr Google for advice. However, these internet sources do not always have pre-appraisal evidence checks, and wikis, meanwhile, have little control over the content placed on their web pages.

Therefore, taking a CA approach enables VNs – and other veterinary professionals – to analyse new information and inform clients of relevant news as an opportunity to deliver important factual information that may be valuable advice for their pet. It is also a very useful way of sifting through data and understanding new products, while discussing new pharmaceutical offers with drug representatives.

Critiquing research papers is a skill that develops with each paper you appraise, and a journal club setting is an excellent way to start. In a journal club you are reviewing literature to find quality evidence to inform nurse decision-making. It allows us to constantly review the information we base our judgements on and, while improving patient care, client education and nursing practices.