Finding a random set of citations in EndNote

Have you ever been asked to find a random set of citation from EndNote? This happens most often to me when researchers are testing out screening procedures, and want to ensure they are all interpreting the screening guidelines the same way. The researchers will all screen the same random set of 10-20 articles and compare results before screening the entire set.

So: what’s the best way to go about this? Sorting from a-z on any given field and selecting the top 10-20 articles isn’t likely to be truly random. For example, sorting by date will retrieve only very new or old articles. Sorting by record number is one possible way to do it, but also isn’t truly random as it will retrieve articles added to the database most or least recently.

Here’s how I take a truly random sample of citations from EndNote.

First, create an output filter in EndNote

The output filter will include only the citation record numbers. Don’t worry, you only have to do this once, and in the future it will all be set up for you!

1. In EndNote, go to Edit –> Output Styles –> New Style
2. In the resulting screen, click “templates” under the heading “bibliography”
3. Then, put your curser in the box below “generic”. Then, click “insert field” –> “Record Number” –> then press enter so that you curser goes to the next line in the text box.
4. Go to “file” –> “save as” and save it to something descriptive like “record-number-only”.

Next, export your record numbers.

1. Back in the main EndNote screen, click the dropdown box at the top of the screen, then “select another style”, and search for your previously created Output Style.
2. Then click “choose”. Ensure that your output style name is displaying in the dropdown box!
3. Select “all references” to make sure all your references (that you want to create a subset from) are displayed. Then click one of the references and press ctrl + a (or cmd + a on a mac) to select all references.
4. Right-hand click and select “copy formatted”.

Create your random subset!

1. Open excel, and press ctrl + v (or cmd + v on a mac) to paste all your record numbers.
2. in the cell to the right of your first record number, insert the formula =rand(). This will create a random number from 0 to 100.
3. Hover the cursor over the bottom-right corner of the cell until it makes a cross. Then click and drag all the way down to the last row that contains a record number
   Insert a row at the top and click “sort & filter” –> “filter” on the menu bar.
4. Then, sort the second row (with the random numbers) from smallest to largest (or largest to smallest).
5. You now have a randomly sorted list! Select and copy the top x number of cells in the first column (however large you want your sample to be).

Format your record numbers to put back into EndNote.

1. Paste your subset of record numbers into word (paste as text, not a table!)
2. Click “replace” on the main toolbar to bring up the find and replace box.
3. Beside the box “find what”, write ^p (the up-carrot symbol followed by “p”).
   Beside the box “replace with”, insert a semi-colon followed by one space.
4. Then click “replace all”.
5. You should have a string of record numbers separated by semi-colons.

Put them back into EndNote!

1. Go back to your EndNote Library.
2. Right-hand click in the sidebar and select “create smart group”
3. Give it a nice title, like “random set” 😃
4. In the first dropdown box, select “record number”, then “word begins with”, then paste in your formatted record numbers separated by semi-colons.
5. Click “create”.
6. All done!

I hope you found this useful. It might sound complicated, but this process really only takes a few seconds once you have gone through it a few times.

Do you have a more efficient or a different way of doing it? What kinds of formatting and database problems do you come across in your position? Feel free to send me a message or tweet at me.

Til next time,
Amanda

using ovid faster and smarter

Did you know that Ovid’s search bar can be used like a command line? Its most common use is to type in search queries, but it can also be used to execute several time-saving commands.

Each command is preceded by two dots (..). These are what tell the database that you don’t want to search for terms, but do something different. Remember that there is no space between the two dots (..) and the command!

Part 1: Save and execute searches

• ..sv ps(search name) will save your search permanently. For example, “..sv ps(Heart-Disease)” (without the quotes) to save the current search. The parenthesis are important — without them, the search will only be saved temporarily (24 hours). I like to periodically type in the same command above while working to save any updates to the search that I’m working on.
• ..e <saved search name> will execute a search. For example, if you have a saved search called Heart-Disease, type “..e Heart-Disease” (without the quotes) to execute the search.
• ..pg all to clear the search history. If your search is saved, it will stay saved, but this allows you to clear the slate and start something new. Similarly, use “.. pg #,#” (without the quotes) to purge specific lines.
• ..dedup # to remove any duplicates from a specific line in the search history.
• ..ps to view the entire search history in a printable format

Part 2: Look up information about MeSH

• ..scope <subject heading> will look up the scope note for the indicated subject heading. For example “..scope heart diseases” (without the quotes).
• ..tree <subject heading> will look up the subject heading in the tree hierarchy. For example, “..tree heart diseases” (without the quotes).
• ..sh <subject heading> to look up the subheading selection window for the subject heading.

(Note: The three commands above can be used with out without the dot dot (..) syntax preceding the command. I like to use it for all commands for consistency).

All of this information is also contained in Ovid’s help documentation.

I hope you find these commands as useful as I do. If you can master these, you’ll be well on your way to becoming a database master (and also wow those around you with you efficient navigating ability!).

Til next time,

Amanda

 

Using a “gold standard” set to test your search

When developing a systematic search, it’s important to use an iterative approach, constantly tweaking and reevaluating your strategy to ensure relevant articles are captured (and hopefully, non-relevant articles are minimised).

Today, I’d like to share a trick that I frequently use when building my searches. First, develop a set of articles which are relevant to your topic. These are articles which should definitely be picked up by your search. The articles might come from researchers or your patrons, other team members on the systematic review, background scoping searches, google scholar, or any other number of places. The more variety in the set of articles, the better. These articles will comprise your “gold standard set” by which you will test your search strategy.

PART 1: Formatting your PMIDs

First, put each of these articles into your citation management system (ideally EndNote). Next, ensure that each article contains a PMID (PubMed ID) in the accession number field (or whichever one you choose). In EndNote, this can often be easily done by clicking “references”, then “find reference updates”. However, do check through all the citations for any that are missed; it may be necessary to manually find the PMID in PubMed.

After you have your gold set all tidied up in EndNote, export the set of references using a custom output filter containing only the accession number field. To set this up in EndNote v7 (only required the first time you do this!):

1. go to Edit -> Output Styles -> New Style.
2. in the sidebar, find “Bibliography” heading and click the “Templates” subheading.
3. in the box that says “Generic”, click “Insert Field”, then “Accession Number”. Save and close your output filter with a descriptive name such as “PMID”.

To export the references using your new filter, first make sure that your newly created output filter is selected (the name should appear in the dropdown box on the top header; if not select the dropdown box, then “select another style”). Next, press ctrl + A to select all references, then right-hand click and select “copy formatted”.

Open a word document and press ctrl + v to paste your formatted references. Your document ought to contain a list of PMIDs – one per line. From here, I use the find and replace tool to automatically format the list of PMIDs for Ovid Medline:

1. Click “find and replace”.
2. In the “find what” box, enter ^p (this stands for the paragraph character)
3. In the “replace with box”, enter “_OR_” (the underscores represent spaces)
4. Press “Replace all”.

Okay! Still with me? Your word document should be formatted most of the way. Now, I finish by adding an open parenthesis at the beginning of the document and replacing the final ” OR sequence with ).ui. The .ui at the end refers to the Ovid Medline field code for accession number (where the PMID is stored). The text of your document should now look something like this:

(“19901971” OR “22214755” OR “22214756” OR “24169943” OR “24311990” OR “18794216” OR “25491195” OR “16931779” OR “9727760” OR “22529271” OR “18757621” OR “25536072” OR “24838102” OR “25025477” OR “23460252” OR “26888209” OR “24381228” OR “25154608” OR “21889426” OR “24165853” OR “25315132” OR “26819213” OR “26936902” OR “27492817” OR “27531721” OR “27522246” OR “27067893”).ui

This process might take a little while to set up the first time, but once everything is automated through your custom output file, it will only take a few seconds in the future. I’m a big fan of front-loading my work to make things easier down the line.

PART 2: Testing your gold standard set

Now, navigate to your draft search strategy in Ovid Medline and paste the full query from part 1 into a new line below the search.

Take the line of your final search results and the line containing your gold standard set and OR them together. If the last two lines in Ovid contain the same number, you’re in luck! All the citations in your gold standard set will be picked up in your draft search. If not, NOT out your original search results to see which ones have been missed; by looking at these citations, you can strategise ways to pick up articles with similar wording or indexing.

OR together your “gold standard” set with your final search results. If the number stays the same, all your gold standard articles are contained in the search strategy.

I sometimes find that researchers are concerned about whether the relevant articles they have found will be captured by my search strategies, so I sometimes include this “gold standard search” in draft strategies that I send. I also annotate my process to make it more clear.

The beauty of this method is that as new relevant papers are discovered from additional sources, you can add them to the gold standard set, and continually check your strategy throughout the drafting process.

How to make screening less painful

You know that feeling when you are running searches for a patron, and want to pick out some of the most relevant papers for them, but it’s a Friday afternoon and your eyes are tired and zomg screening is the worst?

I’ve got a tip to make this process marginally less awful. This life-saving tip comes from a wonderful colleague at the College of Physicians and Surgeons of British Columbia.

First, run your search(es) and download your citations into EndNote (or another citation management program)* for screening. Generally, I only use this tip for general scoping (not for systematic review screening), so I usually end up downloading less than 200 citations for this process, and sometimes as few as 30.

Next, export your citations into a .rtf format with an export filter that includes both the citation information and the abstract. To do this in EndNote (v7), go to the dropdown menu at the top of the page and choose “select another style…”, then search for “annotated”. Click the one with the category “generic”, then click “choose”. You will notice that the preview pane for each citation now contains the citation’s information and its’ abstract.

EndNote dropdown menu and preview pane

Next, export your references by clicking the blue arrow on the top bar. First, press ctrl + a to select all the references. Then save the filetype as .rtf and select “annotated” as your output style. Save the file wherever, then navigate to that folder and open it. It should automatically open in Microsoft Word (or the word processing program of your choice). The file should contain all of your references, with abstracts.

exporting your references from EndNote

Now comes the fun bit! Press ctrl + H (or click “replace” in the main top bar). Under “find what”, type one of the main terms for your first concept. Then click anywhere in the “replace with” box, but instead of typing anything, click “More >>” to expand the options, then click the “format” dropdown box, then “highlight”. The word “highlight” ought to appear below the “replace with box”.

Find and Replace in Microsoft Word

Still with me? Okay. Click “replace all”. Repeat this step with other terms that might be found in the titles and abstracts of the citations (but only for your first concept!). Once you have reached relative saturation, click the highlighter icon in the main top bar, and select a different highlighter colour. Next, repeat the same process as above with your second main concept, until you have reached relative saturation.

Ta da! At this point, you ought to have a pretty colour-coded document which helps you easily see the main concepts from your search. Screening this word document will be much less straining on the eyes and take less time because the main concepts have already been identified for you.

Word document, ready for screening

This trick works better for some topics than others. My example above which uses the concepts of caring and attachment works pretty well. However, complex interventions or other areas with ever-changing terminology might not work as well.

Pro tip: in some cases, it is useful to send this colour coded document to your patron, and let them make decisions about what citations are relevant.

Another pro tip: instead of formatting with a highlighter, which only comes in garish colours (why, microsoft? why??), you can also format the text in any way you want. For example, you can put the relevant terms in bold or italics, or make the text itself different colours.

That’s it for today. Have you ever done this, or something similar? Do you have any protips for screening more quickly and efficiently? Send them to me on Twitter or through the Contact Me form!

* But seriously, if you’re not using EndNote, get on that.

Phrase searching in PubMed is weirdly complicated

In this weeks’ episode of expert searching: hubris edition, I found that I don’t understand PubMed nearly as well as I thought. I’ll confess to primarily being an Ovid user myself. I’ve never found the PubMed interface intuitive. I dislike not seeing my search history on the same page as my results, reading the search history from bottom to top of the page, and not being able to use proximity operators.

But, as an information specialist, one sometimes has to use platforms that one does not like.

In the course of using PubMed last week, I found that my search queries were not being interpreted the way I had intended. My query in Ovid retrieved over 9000 results, but when translated (as accurately as one can translate between the two…), my search retrieved less than half the results!

The solution: When search terms are in quotation marks, PubMed ignores the truncation symbol. My strategy had relied heavily on truncated phrases, all of which were in quotation marks (to avoid PubMed’s automatic term mapping), and all of which were being interpreted as singular rather than plural terms (e.g. “patient outcome*” would search only for “patient outcome” and not “patient outcomes”).

To demonstrate:

Search	Query	Items found
#3	Search patient reported outcome*[Title/Abstract]	7799
#2	Search patient reported outcome[Title/Abstract]	3319
#1	Search “patient reported outcome*”[Title/Abstract]	3319

Oh bother! Why can’t databases just read my mind already?!

This bug(?) in PubMed’s system of interpreting logic brings up a few important issues for systematic searching. I spent some time this week figuring out how the system works.

PubMed’s interpretation of the unqualified search: patient reported outcomes. Ick! That’s not what I wanted at all!

PubMed’s automatic term mapping kicks in when no truncation, quotation marks, or field tags (eg [tiab]) are used (an unqualified search). In the case of an unqualified search, PubMed searches for terms within MeSH, authors, journals, and the phrase index. If none are found, PubMed starts searching for the individual words within a phrase and adding them to your search. To see how PubMed interprets your search query, see the “search details” box in the right-hand sidebar on the search results page. This will show if any automatic term mapping was used, and if so, how.

The main take-away from this experience for me is:

To conduct a replicable and transparent search in PubMed, always in ensure that your search terms and phrases are either: 1) in quotation marks, 2) use truncation, or 3) in the phrase index. Also, never use both quotation marks and truncation at the same time. Otherwise, you run the risk of having your beautifully constructed search destroyed by silly computer logic.

The secret to bibliometric analysis: generating a list of PMIDs

By now, it’s probably no secret that I love crunching bibliometric data. I find that analysing my results — both during search strategy formation and after downloading final results — gives me a broader perspective and see trends that I might otherwise miss.

However, analysing data can sometimes be time consuming and clunky. Data never seems to be in the format that you want it when you need it; the precise tool that you need at that moment hasn’t been invented yet or is otherwise proprietary; the right software for the job requires a programming language you haven’t yet learned, and so forth. Sometimes you want a quick and dirty answer to help develop a strategy and it doesn’t have to be tidy or perfect, but you need it now!

Here’s my quick and dirty trick for analysing your bibliometric [medline] data:

1. Generate a list of PMIDs from your results (whether your strategy is finalised or not!)
2. Pop into the data analysis program of your choosing…

The beauty of this trick is that you can copy-paste whatever you are working on at this very moment (provided you’re working with medline data, of course…) and get real-time feedback. No need to mess with clunky software interfaces or retype your strategy.

Generate a list of PMIDs

If you’re using PubMed, this part is easy. Click the “Format: Summary” drop down menu just below the search bar, then select “PMID”. Et voila! The resulting page is a plain text list of PMIDs, taken from the results on the previous page.

Note that the resulting PMID list will show only the citations from the previous page, so you may want to scroll to the bottom of the screen to show the max number of citations per page (200 at the time of this writing).

If you’re working in Ovid (like I generally do), this is a bit trickier. Ovid citations still contain a PMID in the “unique identifier” field, but it’s not quite so easy to extract. There’s a few ways to go about this, but my usual strategy is to:

1. Download all search results into EndNote reference management software
2. Extract a list of PMIDs through a custom export filter

The downside to this method is that there is a limit to the amount of citations you can export at once from Ovid, and EndNote also gets a wee bit finicky when you start importing citations by the thousands… Thus, this technique is best done with small-ish citation sets.

Analyse your data

Once you have your list of PMIDs, you can pop them into a variety of different tools to crunch the data in different ways. For example, try pasting your list into:

• PubReminer  – for a word count analysis of authors, journals, MeSH, title/abstract words…
• Medline Trends – for an analysis of citations over time
• GoPubMed – for a variety of filters (maps! bar graphs! frequency charts!)
• Yale MeSH Analyser – for a side-by-side comparison of MeSH usage

And more! Someday I intend to write up a full list of medline data analysis tools freely available online, but that day is not today…

It’s not necessary to input a full search strategy into most bibliometric analysis programmes… simply paste in your PMIDs!

Why would a person bother to do this?

Building a search strategy is an iterative process and it requires using a lot of different tools. For example, you can use your own common sense and intuition, but other tried-and-true strategies include: backwards/forwards citation chaining, talking to experts in the field, or looking at highly cited papers/journals in the field.

Using quick data analysis strategies throughout the process of building a search strategy will help ensure that important concepts aren’t missed. They provide a more objective picture of what’s happening, what’s missing, and how you can better refine your strategy.

That’s it for this week!

PS This is my first proper blog and I must say… keeping a blog up to date is not as easy as I thought. Please do let me know if you find this content useful and I will try my utmost to keep ’em coming! You can use the site contact form or find me on twitter at @v_woolf.

 

ONE WEIRD TRICK THAT OVID WON’T TELL YOU

No, it’s not a food that will give you a slimmer stomach or boost your manly prowess.

I’m talking, of course, about the ability to find the total number of citations in the Medline database. Why on earth would someone want to know how many citations are in a database, you ask?

• To compare and contrast the size with other databases
• For FUN, because you’re a nerd like me
• Um… because?

It’s relatively straightforward to find the total number of citations in PubMed. Their documentation helpfully tells us: “To search for the total number of PubMed citations, enter all [sb] in the search box.”

However, a few days ago I was struck with an awkward problem. I needed to find the total number of citations in Ovid Medline. Why? I had conducted a straightforward scoping search for a researcher and created a basic frequency analysis of the number of citations retrieved in the search per year to show the publishing trends in the topic over time.

frequency analysis, non-normalised (raw count of citations)

The researcher asked me to normalise the data…. say what?? Do I look like a statistician?

I knew I couldn’t use the numbers from PubMed, because the two have slight differences in content. And I couldn’t translate my strategy into PubMed because it relied heavily on the adjacent operator (which is absent in PubMed).

After some frantic searching, I found out that this was not such a difficult task: all I needed to do was take the number of citations retrieved from the search in a given year, and divide this by the number of total citations published in the database that year. This would even out any potential errors in the chart from anomalies in the database as a whole.

The problem: I could not find an equivalent operation in Ovid Medline to PubMed’s all[sb] command. After combing through Ovid’s documentation, I finally broke down and tweeted them… and received a response within a few hours.

@WKHealthOvid is there a way to retrieve total citations in ovid medline, similar to all[sb] in pubmed? thank you! 🙂

— Amanda Wanner (@v_woolf) October 18, 2016

@v_woolf Hi Amanda, you can use docz.dz on the command line to do this. Let me know if that works!

— Ovid (@WKHealthOvid) October 18, 2016

I know everyone’s been waiting with bated breath to find the answer: it’s docz.dz.

What does the .dz field code stand for? No idea. But anyway, it seems to get the trick done, and now I have my nicely normalised graph. In the second image, below, you can see that the downtick in citations for the year 2016 has vanished, because the number of citations retrieved from the search is proportionate to the total citations published this year.

 

frequency analysis, normalised (results as a percentage of total citations in database)

Happy story! The end.

PS Cheers to Ovid’s social media team! They are totally on the ball.