So, you’ve got a literature review to write and the deadline is approaching. Perhaps you don’t know how to begin, or maybe you’ve started and got stuck. Perhaps you’ve used one of the many discovery tools out there to find relevant papers, but now you have to read and filter them all. You’ve got a folder full of articles, and you’re learning that PDF doesn’t actually stand for ‘print, download, forget’….
As graduates and post-graduates ourselves, we’ve been there too. And it’s why we built Scholarcy. In this post, our goal is to help you learn some new techniques to quickly evaluate those papers you downloaded.
Now, much has been written on how to search the literature1, and there are some great guides on how to organise your sources and how to structure your writing2. But when they suggest you identify gaps or weaknesses in the literature, or areas of controversy, how do you actually do this? While there are some excellent guides3 on how to read a paper4, they tell you what to do, rather than how to do it. When they suggest you verify the paper’s main assumptions, theoretical frameworks, and contributions, how do you go about doing that?
Interestingly, some recommend not reading the abstract5, as this can provide a biased interpretation of the paper, given that author will often write the abstract in such a way as to show the paper in the best light. Jennifer Raff5 recommends reading the introduction and background first to identify:
- what the main problem is that the researchers are trying to solve
- prior work on the subject and its possible limitations
In the background
With Scholarcy’s flashcard technology, getting this background context is a simple process, as we make all citations clickable:
Clickable citations in Scholarcy
This will take you direct to the source in your browser, where you can use our Chrome Extension to highlight potential limitations instantly:
Scholarcy identifies study limitations
Keshav4 suggests looking up background reading for information on unfamiliar terms or acronyms. In Scholarcy, acronyms are automatically expanded, and background knowledge to get you up to speed is just one click away:
Follow the citation trail
Keshav also recommends following up references if you are unfamiliar with the topic. In the Scholarcy flashcard, this is super easy, as you can export all references to Zotero, which will locate the full texts for you, or you can click on the LibKey or UnPaywall links for individual references:
It’s all in the method
Next, Raff advocates studying the methods in detail. What was done and with whom? Our flashcards help with this by isolating mentions of study participants and data analysis techniques used:
If you are not sure what specific methods such as Kruskal-Wallis, Benjamini-Hochberg or Dunn’s tests are, you can get more information on them by clicking on them. We can see that the Benjamini-Hochberg test is a procedure to identify the false discovery rate which, if not corrected for, could cause the results to be inaccurately interpreted.
Standing on the shoulders of giants
Both Raff and Keshav suggest comparing the results to previous work in the field. The ‘comparative analysis’ section of the Scholarcy flashcard makes this easier to do – again, all citations are clickable, and our integration with scite.ai shows whether others have confirmed or questioned those findings. Pay attention here though, as some of these may be self-citations, or to work by the same research group as the current study.
In conclusion, discovery tools will get you so far, but for appraising individual studies, AI tools that distil and connect the key information can really accelerate the process. You need more than a summary – you also need background information on the results of previous work, the current findings in the context of what has gone before, and a quick overview of limitations and future work. Is the study is standing on the shoulders of giants? Scholarcy flashcards help you to decide.
References
- vom Brocke, J., Simons, A., Riemer, K., Niehaves, B., Plattfaut, R., & Cleven, A. (2015). Standing on the Shoulders of Giants: Challenges and Recommendations of Literature Search in Information Systems Research. Communications of the Association for Information Systems, 37, pp-pp. https://doi.org/10.17705/1CAIS.03709
- University of Edinburgh (2021). A general guide on how to conduct and write a literature review. Available from: https://www.ed.ac.uk/institute-academic-development/study-hub/learning-resources/literature-review
- BMJ (nd). How to read a paper. Available from: https://www.bmj.com/about-bmj/resources-readers/publications/how-read-paper
- Keshav, S. (nd). How to read a paper. Available from: https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf
- Raff, J. (2016). How to read and understand a scientific paper: a guide for non-scientists. Available from: https://blogs.lse.ac.uk/impactofsocialsciences/2016/05/09/how-to-read-and-understand-a-scientific-paper-a-guide-for-non-scientists/