Meta-analysis - Searching Relevant Studies

Once we have a research question and we have established inclusion and exclusion criteria for relevant studies, we need to determine a search strategy to identify relevant clinical trials.

In developing our search strategy, we should keep in mind that our research should find as many studies as possible, while at the same time it should be efficient. One can try to find all relevant trials ever done on a particular topic, but this is practically impossible and quite inefficient. Generally, the harder one tries to find studies, more relevant studies one will find, but after a certain number of studies are identified, every incremental effort result in a decrease in the number of identified studies. When should one stop searching for additional studies is controversial.

In general, relevant studies are identified by searching medical databases. PubMed is a database containing more than 10 million references, and more than 400,000 references are added annually. It covers more than 3900 medical journals in 40 languages (88% in English). One can put “Limits” to one’s search, which helps to decrease the number of returned references. Its major deficiencies are that it covers only about 33% of medical journals and that it only goes back to 1966. A second database is EMBASE, it is somewhat larger than PubMed, but is commercial and no free version is available. Another important source of randomized controlled trials is “The Cochrane Controlled Trials Register”. This Register includes all randomized controlled trials published in 1700 medical journals. It does not contain non-randomized clinical studies. OVID Online is another database that can be searched. Although OVID is a commercial database, one can access it through Merck Medicus website. One should remember that there is overlap between these databases and most of the articles retrieved will be the same. Other databases that can be searched are AMED, BIOSIS, CINHAL, PsycINFO, and Science Citation Index. An important aspect of search for relevant clinical trials is to perform a hand-search of references of the retrieved articles as well as relevant review articles. This search usually retrieves a significant number of relevant studies that have not been properly indexed by databases.

Whether one decides to include clinical trials that are not (yet) published in medical journals determines the next step in search. If one decides to search unpublished clinical trials, there are multiple resources that can be searched. For example, abstracts from relevant conference proceedings, ClinicalTrials.gov, CRISP database of NIH, or FDA database of clinical trials. Investigators can be contacted individually to learn about ongoing or recently completely but unpublished trials.

If properly done, a comprehensive search of the relevant clinical trials can tremendously improve the quality of the meta-analysis. On the other hand, an incomplete search will result in publication bias (to be discussed later) which can severely compromise the results and conclusion of the meta-analysis.