I’m often approached by masters and PhD students and researchers in my institution to advise on systematic review projects in the early stages. I’ve found that the skill levels to complete a systematic or scoping review are variable, and that many researchers need a primer to get up to speed about the process of conducting a review, what skills are required, and in particular, how to go about the planning process.
I support many projects in depth from start to finish, but for many projects at my institution, I only have the time to provide advice and consultations. Unfortunately, I quickly learned that throwing a lot of information at people in a short period of time was not useful, and I would sometimes see the same researchers at a later consultation who hadn’t gotten very far with their projects and needed a lot of the same information again.
There are many, many resources online for conducting review projects, including some enviable LibGuides (I personally like the Queens University knowledge synthesis guide and University of Toronto learning to search guide). However, I wanted a resource that I could use when physically sitting with someone in a meeting room, where we could plan out their review project together. And I was getting pretty tired of drawing the same venn diagrams of how “AND” and “OR” boolean operators work on whatever scratch paper I had handy.
I recently developed a guide that fits these purposes, and after a few iterations and some testing and feedback, I’ve put it online for others to use and edit as they wish with a CC-NC-SA 4.0 License. The goal of this guide is to provide a resource that:
Can be printed and used as a workbook to guide a systematic reviews consultation
Also contains enough information to be a stand-alone self-learning resource for after the consultation (e.g. the information on boolean operators)
Is not too long to be intimidating or overwhelming for someone just getting started
Without a doubt, there will be further refinements and additions to the guide over time, but for now, please feel free to download, use, and edit for your own purposes. Any feedback or comments are also gratefully accepted. 🙂
In the “multi-line” vs “single line” searches debate, one point that is often thrown around is: multi-line searches are more cumbersome to edit and run. Even with Ovid’s new “edit” button, it still takes a few clicks and a few page refreshes to edit a strategy and see the results. When making lots of changes quickly to a strategy, this time can really add up.
One underappreciated and little known tool is Ovid’s mutli-line launcher. It’s beautiful! The multi-line launcher allows a user to copy/paste a multi-line strategy directly into the search box, press enter, and view the search results – with hits for each line – as normal.
When making edits to a strategy I tend to do the following:
paste the strategy into the multi-line launcher box
ensure that the line numbers are still correct or changed if needed
Did you know that Ovid’s search bar can be used like a command line? Its most common use is to type in search queries, but it can also be used to execute several time-saving commands.
Each command is preceded by two dots (..). These are what tell the database that you don’t want to search for terms, but do something different. Remember that there is no space between the two dots (..) and the command!
Part 1: Save and execute searches
..sv ps(search name) will save your search permanently. For example, “..sv ps(Heart-Disease)” (without the quotes) to save the current search. The parenthesis are important — without them, the search will only be saved temporarily (24 hours). I like to periodically type in the same command above while working to save any updates to the search that I’m working on.
..e <saved search name> will execute a search. For example, if you have a saved search called Heart-Disease, type “..e Heart-Disease” (without the quotes) to execute the search.
..pg all to clear the search history. If your search is saved, it will stay saved, but this allows you to clear the slate and start something new. Similarly, use “.. pg #,#” (without the quotes) to purge specific lines.
..dedup # to remove any duplicates from a specific line in the search history.
..ps to view the entire search history in a printable format
Part 2: Look up information about MeSH
..scope <subject heading> will look up the scope note for the indicated subject heading. For example “..scope heart diseases” (without the quotes).
..tree <subject heading> will look up the subject heading in the tree hierarchy. For example, “..tree heart diseases” (without the quotes).
..sh <subject heading> to look up the subheading selection window for the subject heading.
(Note: The three commands above can be used with out without the dot dot (..) syntax preceding the command. I like to use it for all commands for consistency).
I hope you find these commands as useful as I do. If you can master these, you’ll be well on your way to becoming a database master (and also wow those around you with you efficient navigating ability!).
When developing a systematic search, it’s important to use an iterative approach, constantly tweaking and reevaluating your strategy to ensure relevant articles are captured (and hopefully, non-relevant articles are minimised).
Today, I’d like to share a trick that I frequently use when building my searches. First, develop a set of articles which are relevant to your topic. These are articles which should definitely be picked up by your search. The articles might come from researchers or your patrons, other team members on the systematic review, background scoping searches, google scholar, or any other number of places. The more variety in the set of articles, the better. These articles will comprise your “gold standard set” by which you will test your search strategy.
PART 1: Formatting your PMIDs
First, put each of these articles into your citation management system (ideally EndNote). Next, ensure that each article contains a PMID (PubMed ID) in the accession number field (or whichever one you choose). In EndNote, this can often be easily done by clicking “references”, then “find reference updates”. However, do check through all the citations for any that are missed; it may be necessary to manually find the PMID in PubMed.
After you have your gold set all tidied up in EndNote, export the set of references using a custom output filter containing only the accession number field. To set this up in EndNote v7 (only required the first time you do this!):
go to Edit -> Output Styles -> New Style.
in the sidebar, find “Bibliography” heading and click the “Templates” subheading.
in the box that says “Generic”, click “Insert Field”, then “Accession Number”. Save and close your output filter with a descriptive name such as “PMID”.
To export the references using your new filter, first make sure that your newly created output filter is selected (the name should appear in the dropdown box on the top header; if not select the dropdown box, then “select another style”). Next, press ctrl + A to select all references, then right-hand click and select “copy formatted”.
Open a word document and press ctrl + v to paste your formatted references. Your document ought to contain a list of PMIDs – one per line. From here, I use the find and replace tool to automatically format the list of PMIDs for Ovid Medline:
Click “find and replace”.
In the “find what” box, enter ^p (this stands for the paragraph character)
In the “replace with box”, enter “_OR_” (the underscores represent spaces)
Press “Replace all”.
Okay! Still with me? Your word document should be formatted most of the way. Now, I finish by adding an open parenthesis at the beginning of the document and replacing the final ” OR sequence with ).ui. The .ui at the end refers to the Ovid Medline field code for accession number (where the PMID is stored). The text of your document should now look something like this:
(“19901971” OR “22214755” OR “22214756” OR “24169943” OR “24311990” OR “18794216” OR “25491195” OR “16931779” OR “9727760” OR “22529271” OR “18757621” OR “25536072” OR “24838102” OR “25025477” OR “23460252” OR “26888209” OR “24381228” OR “25154608” OR “21889426” OR “24165853” OR “25315132” OR “26819213” OR “26936902” OR “27492817” OR “27531721” OR “27522246” OR “27067893”).ui
This process might take a little while to set up the first time, but once everything is automated through your custom output file, it will only take a few seconds in the future. I’m a big fan of front-loading my work to make things easier down the line.
PART 2: Testing your gold standard set
Now, navigate to your draft search strategy in Ovid Medline and paste the full query from part 1 into a new line below the search.
Take the line of your final search results and the line containing your gold standard set and OR them together. If the last two lines in Ovid contain the same number, you’re in luck! All the citations in your gold standard set will be picked up in your draft search. If not, NOT out your original search results to see which ones have been missed; by looking at these citations, you can strategise ways to pick up articles with similar wording or indexing.
I sometimes find that researchers are concerned about whether the relevant articles they have found will be captured by my search strategies, so I sometimes include this “gold standard search” in draft strategies that I send. I also annotate my process to make it more clear.
The beauty of this method is that as new relevant papers are discovered from additional sources, you can add them to the gold standard set, and continually check your strategy throughout the drafting process.
By now, it’s probably no secret that I love crunching bibliometric data. I find that analysing my results — both during search strategy formation and after downloading final results — gives me a broader perspective and see trends that I might otherwise miss.
However, analysing data can sometimes be time consuming and clunky. Data never seems to be in the format that you want it when you need it; the precise tool that you need at that moment hasn’t been invented yet or is otherwise proprietary; the right software for the job requires a programming language you haven’t yet learned, and so forth. Sometimes you want a quick and dirty answer to help develop a strategy and it doesn’t have to be tidy or perfect, but you need it now!
Here’s my quick and dirty trick for analysing your bibliometric [medline] data:
Generate a list of PMIDs from your results (whether your strategy is finalised or not!)
Pop into the data analysis program of your choosing…
The beauty of this trick is that you can copy-paste whatever you are working on at this very moment (provided you’re working with medline data, of course…) and get real-time feedback. No need to mess with clunky software interfaces or retype your strategy.
Generate a list of PMIDs
If you’re using PubMed, this part is easy. Click the “Format: Summary” drop down menu just below the search bar, then select “PMID”. Et voila! The resulting page is a plain text list of PMIDs, taken from the results on the previous page.
Note that the resulting PMID list will show only the citations from the previous page, so you may want to scroll to the bottom of the screen to show the max number of citations per page (200 at the time of this writing).
To extract PMIDs from Ovid:
select all citations (or a range if there’s a lot!)
select “excel” under the drop-down menu “Export To:”
select “custom fields”
under “select fields” (beside the “custom fields” radio button), unselect everything except “unique identifier” (this is the field that contains the PMID in Ovid)
Then select “export citations”
An excel file should download with a column of PMIDs, which can then be copied/pasted.
(Thanks to Michelle Fiander for the excel tip!)
Analyse your data
Once you have your list of PMIDs, you can pop them into a variety of different tools to crunch the data in different ways. For example, try pasting your list into:
PubReminer – for a word count analysis of authors, journals, MeSH, title/abstract words…
And more! Someday I intend to write up a full list of medline data analysis tools freely available online, but that day is not today…
Why would a person bother to do this?
Building a search strategy is an iterative process and it requires using a lot of different tools. For example, you can use your own common sense and intuition, but other tried-and-true strategies include: backwards/forwards citation chaining, talking to experts in the field, or looking at highly cited papers/journals in the field.
Using quick data analysis strategies throughout the process of building a search strategy will help ensure that important concepts aren’t missed. They provide a more objective picture of what’s happening, what’s missing, and how you can better refine your strategy.
That’s it for this week!
PS This is my first proper blog and I must say… keeping a blog up to date is not as easy as I thought. Please do let me know if you find this content useful and I will try my utmost to keep ’em coming! You can use the site contact form or find me on twitter at @v_woolf.
However, a few days ago I was struck with an awkward problem. I needed to find the total number of citations in Ovid Medline. Why? I had conducted a straightforward scoping search for a researcher and created a basic frequency analysis of the number of citations retrieved in the search per year to show the publishing trends in the topic over time.
The researcher asked me to normalise the data…. say what?? Do I look like a statistician?
I knew I couldn’t use the numbers from PubMed, because the two have slight differences in content. And I couldn’t translate my strategy into PubMed because it relied heavily on the adjacent operator (which is absent in PubMed).
After some frantic searching, I found out that this was not such a difficult task: all I needed to do was take the number of citations retrieved from the search in a given year, and divide this by the number of total citations published in the database that year. This would even out any potential errors in the chart from anomalies in the database as a whole.
The problem: I could not find an equivalent operation in Ovid Medline to PubMed’s all[sb] command. After combing through Ovid’s documentation, I finally broke down and tweeted them… and received a response within a few hours.
@WKHealthOvid is there a way to retrieve total citations in ovid medline, similar to all[sb] in pubmed? thank you! 🙂
I know everyone’s been waiting with bated breath to find the answer: it’s docz.dz.
What does the .dz field code stand for? No idea. But anyway, it seems to get the trick done, and now I have my nicely normalised graph. In the second image, below, you can see that the downtick in citations for the year 2016 has vanished, because the number of citations retrieved from the search is proportionate to the total citations published this year.
Happy story! The end.
PS Cheers to Ovid’s social media team! They are totally on the ball.
My inaugural tip for the Expert Searching blog comes, fittingly, through a chain of colleagues passed down mentor to mentee. I believe this tip originates from the irreplaceable Dean Guistini of HLWiki.
Ovid Medline recently added a feature to add search strategy annotations, but it’s clunky and annoying. To add annotations, you have to click several times, and to top it off, they aren’t even visible while constructing and executing the search. How useless is that?
However, there’s a secret nobody has told you: it’s always been possible to add annotations to your searches! Simply add square brackets to the end of any line. Any text inside the square brackets is meant to be read by people only; the computer disregards this content. These in-text annotations are a useful way to document the search process and to see what sets of concepts you are combining.
Another way to use the square brackets are to add them to a line all by themselves. This helps separate parts of the search very clearly. If you’re testing out lots of different terms and combining concepts all over the place, it’s a good way to look back on your work and see what’s going on.
Why annotate your work?
Others will be able to understand your search strategy
You will be able to understand your search strategy!
It shows your thought process and rationale for making different decisions
It makes everyone happy because it doesn’t look like gibberish