91

What belongs in a defensible systematic review workflow?. Build the thread as a staged workflow: define the research question, set eligibility criteria, search, screen, appraise bias, decide between narrative synthesis and meta-analysis, assess heterogeneity, and report transparently. Emphasize decision points where poor documentation or unsuitable pooling can weaken the review.

Staged systematic review workflow

The two attached sources describe systematic review as a staged process that starts with a clearly defined question and preset eligibility criteria, then moves through searching, screening, extraction, quality appraisal, synthesis choice, heterogeneity assessment, and transparent reporting.[1][2] The core validity risk is not only whether the review is comprehensive, but whether each decision is documented well enough that readers can see why studies were included, why they were pooled, and when pooling was avoided.[3][4]

Recommended workflow and key decision points

  1. Frame the review question first, because it determines study identification, eligibility criteria, data extraction, and synthesis.[5] Use an appropriate framework such as PICO, PICOTTS, SPIDER, SPICE, or ECLIPSE when it fits the review type.[6][7]
  2. Set inclusion and exclusion criteria before the search begins, so selection is not shaped by what is found later.[8][9]
  3. Run a comprehensive search using synonyms, MeSH terms, Boolean operators, truncation, wildcards, and proximity operators, and validate the search against expected relevant studies.[10]
  4. Remove duplicates, then screen titles and abstracts, followed by full-text eligibility checks, ideally with at least two independent reviewers and a PRISMA flow diagram.[11]
  5. Extract outcomes, study characteristics, and unit-conversion details, and predefine how missing or unclear information will be handled.[12]
  6. Appraise risk of bias or methodological quality using tools matched to the study design, with separate reviewers when possible.[13]
  7. Choose synthesis based on the data, using narrative or qualitative synthesis when studies cannot be pooled and meta-analysis only when statistical pooling is appropriate.[14][15][16]
  8. Select an effect-size metric that matches the data, and apply a weighting scheme that is defensible for the available variance information.[17]
  9. Before pooling, assess heterogeneity visually and statistically, quantify inconsistency with H2 and I2, and explore sources with subgroup analysis, meta-regression, Baujat plots, or sensitivity analyses such as single-study or cumulative exclusion.[18][19][20][21][22]
  10. Check for publication bias with funnel plots, but interpret funnel asymmetry cautiously because it can reflect heterogeneity, selective reporting, chance, or other bias rather than publication bias alone.[23][24][25]
  11. Report pooled estimates, confidence intervals, forest plots, funnel plots, and limitations, and interpret results in light of study quality, bias, and the strength of the evidence.[26][27]

Decision table: bias, heterogeneity, pooling, and sensitivity

DomainWhat to checkResponse or correctionAssumption or caution
Publication biasInspect funnel plots for asymmetry[28][29][30]Document the check and interpret conservatively[31][32]Funnel asymmetry is not proof of publication bias by itself[33][34]
Small-study effectsLook for larger effects in smaller studies[35][36]Consider whether the pattern reflects bias, heterogeneity, or chance[37][38]Do not equate small-study effects automatically with publication bias[39][40]
HeterogeneityUse visual inspection plus H2 and I2[41][42][43]Explore with subgroup analysis, meta-regression, Baujat plots, and exclusion-based sensitivity checks[44][45][46]Pooling is weak when studies are not sufficiently similar[47][48]
Effect-size selectionChoose a metric consistent with the outcome and design[49][50][51]Document transformations and definitions clearly[52][53]Confusing standard deviation with standard error can overestimate effects and narrow confidence intervals[54][55]
Sensitivity analysisTest whether findings change when studies are excluded one at a time or cumulatively[56]Report any influential studies and explain the decision to keep or exclude them[57]If results shift materially, the pooled estimate is not robust[58]
Narrative synthesisAsk whether the evidence is too heterogeneous or otherwise unsuitable for pooling[59][60]Use narrative or qualitative synthesis, including SWiM when appropriate[61][62]Prefer this route when meta-analysis would be inappropriate[63][64]

What weak documentation most threatens validity

  • A vague or late-redefined question can invite data dredging and post hoc interpretation.[65][66]
  • A search that is narrow, unvalidated, or poorly documented can miss studies and introduce selection bias.[67][68]
  • Incomplete extraction of sample sizes, uncertainty estimates, or unit conversions can distort weighting and effect estimates.[69][70]
  • Weak risk-of-bias appraisal can let low-rigor or expectation-biased studies dominate the synthesis.[71][72]
  • Treating exploratory subgroup or meta-regression findings as causal conclusions goes beyond what the sources support.[73]
  • Skipping heterogeneity and bias diagnostics, or omitting the rationale for exclusions, weakens transparency and reproducibility.[74][75]

Conclusion

A defensible review workflow is one that fixes the question and eligibility rules up front, searches comprehensively, screens and extracts data transparently, appraises bias with design-appropriate tools, and only pools studies when the effect size, weighting, and heterogeneity all support it.[76][77][78][79] If the evidence is too heterogeneous or the assumptions for pooling are weak, the safer choice is narrative synthesis or SWiM rather than forcing a meta-analysis.[80][81]


Related Content From The Pandipedia