Skip to main content

Learning to Rank Claim-Evidence Pairs to Assist Scientific-Based Argumentation

  • Conference paper
  • First Online:
Digital Libraries for Open Knowledge (TPDL 2019)

Abstract

We consider the novel problem of learning to rank claim-evidence pairs to ease the task of scientific argumentation. Researchers face daily scientific argumentation when writing research papers or project proposals. Once confronted with a sentence that requires a citation, they struggle to find the manuscript that can support it. In this work, we call such sentences claims – a natural language sentence – that needs a citation to be credible. Evidence in our work refers to a paper that provides credibility to its corresponding claim. We tackle the scientific domain where the task of matching claim-evidence pairs is hindered by complex terminology variations to express the same concept and also by the unknown characteristics beyond content that makes a paper worth to be cited. The former calls for a suitable representation capable of dealing with the challenge of content-based matching considering domain knowledge, whereas the latter implies a need to propose semantic features of suitable characteristics to guide the learning task. To this end, we test the scope and limitation of a deep learning model tailored to the task. Our experiments reveal what specific attributes can guide the learning task, the impact of using domain knowledge in the form of concepts and also the assessment of which metadata of a document, e.g., ‘background’, ‘conclusion’, ‘method’, ‘objective’, or ‘results’ should be considered to achieve better results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://en.wikipedia.org/wiki/Wikipedia:Citing_sources.

  2. 2.

    https://en.wikipedia.org/wiki/Wikipedia:Verifiability.

References

  1. Aharoni, E., et al.: A benchmark dataset for automatic detection of claims and evidence. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics, pp. 1489–1500. Dublin City University and Association for Computational Linguistics, Dublin (2014)

    Google Scholar 

  2. Bergstra, J., et al.: Algorithms for hyper-parameter optimization. In: Proceedings of the 24th International Conference on Neural Information Processing Systems, pp. 2546–2554. Curran Associates Inc., Lake Tahoe (2011)

    Google Scholar 

  3. Bergstrom, C.T.: Eigenfactor: measuring the value and prestige of scholarly journals. Coll. Res. Libr. News. 68(5), 314–316 (2007)

    Article  Google Scholar 

  4. Bordes, A., Weston, J., Usunier, N.: Open question answering with weakly supervised embedding models. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8724, pp. 165–180. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44848-9_11

    Chapter  Google Scholar 

  5. Dehghani, M., et al.: Neural ranking models with weak supervision. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 65–74. ACM, New York (2017)

    Google Scholar 

  6. Falagas, M.E., et al.: Comparison of SCImago journal rank indicator with journal impact factor. FASEB J. 22(8), 2623–2628 (2008)

    Article  Google Scholar 

  7. Fetahu, B., et al.: Finding news citations for Wikipedia. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 337–346. ACM, Indianapolis (2016)

    Google Scholar 

  8. Garfield, E.: The history and meaning of the journal impact factor. J. Am. Med. Assoc. 295(1), 90–93 (2006)

    Article  Google Scholar 

  9. Guo, J., et al.: A deep relevance matching model for ad-hoc retrieval. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 55–64. ACM, Indianapolis (2016)

    Google Scholar 

  10. Kalchbrenner, N., et al.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 655–665. Association for Computational Linguistics, Baltimore (2014)

    Google Scholar 

  11. Kilicoglu, H., et al.: SemMedDB: a PubMed-scale repository of biomedical semantic predications. J. Bioinform. 28(23), 3158–3160 (2012)

    Article  Google Scholar 

  12. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751. Association for Computational Linguistics, Doha (2014)

    Google Scholar 

  13. Levy, R., et al.: Context dependent claim detection. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics, pp. 1489–1500. Dublin City University and Association for Computational Linguistics, Dublin (2014)

    Google Scholar 

  14. Liu, T.-Y.: Learning to rank for information retrieval. Found. Trends Inf. Retr. 3(3), 225–331 (2009)

    Article  Google Scholar 

  15. Lu, Z., Li, H.: A deep architecture for matching short texts. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, vol. 1, pp. 1367–1375. Curran Associates Inc., Lake Tahoe (2013)

    Google Scholar 

  16. Mikolov, T., et al.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, pp. 3111–3119. Curran Associates Inc., Lake Tahoe (2013)

    Google Scholar 

  17. Mitra, B., Craswell, N.: An introduction to neural information retrieval. Found. Trends Inf. Retr. 13(1), 1–126 (2018)

    Article  Google Scholar 

  18. Rindflesch, T.C., Fiszman, M.: The interaction of domain knowledge and linguistic structure in natural language processing: interpreting hypernymic propositions in biomedical text. J. Biomed. Inform. 36, 462–477 (2003)

    Article  Google Scholar 

  19. Roitman, H., et al.: On the retrieval of Wikipedia articles containing claims on controversial topics. In: Proceedings of the 25th International Conference Companion on World Wide Web, pp. 991–996 International World Wide Web Conferences Steering Committee, Montreal (2016)

    Google Scholar 

  20. Severyn, A., Moschitti, A.: Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 373–382. ACM, Santiago (2015)

    Google Scholar 

  21. Shen, Y., et al.: Learning semantic representations using convolutional neural networks for web search. In: Proceedings of the 23rd International Conference on World Wide Web, pp. 373–374. ACM, New York (2014)

    Google Scholar 

  22. De Vine, L., et al.: Medical semantic similarity with a neural language model. In: Proceedings of the 23rd ACM International Conference on Information and Knowledge Management, pp. 1819–1822. ACM, New York (2014)

    Google Scholar 

  23. Zamani, H., et al.: From neural re-ranking to neural ranking: learning a sparse representation for inverted indexing. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 497–506. ACM, Torino (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to José María González Pinto .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

González Pinto, J.M., Celik, S., Balke, WT. (2019). Learning to Rank Claim-Evidence Pairs to Assist Scientific-Based Argumentation. In: Doucet, A., Isaac, A., Golub, K., Aalberg, T., Jatowt, A. (eds) Digital Libraries for Open Knowledge. TPDL 2019. Lecture Notes in Computer Science(), vol 11799. Springer, Cham. https://doi.org/10.1007/978-3-030-30760-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30760-8_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30759-2

  • Online ISBN: 978-3-030-30760-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics