IDCite is a large-scale multidisciplinary dataset of citation contexts and citation intents spanning 21 Essential Science Indicators (ESI) fields.

The dataset contains 1,857,503 citation records linking 1,467,045 citing papers and 23,479 seed papers, enriched with citation contexts, citation-intent annotations, publication metadata, and normalized scholarly entities.

This repository contains the complete pipeline for metadata collection, citation harvesting, citation-context integration, citation-intent annotation, entity normalization, dataset generation.

• Citation events: 1,857,503
• Citing papers: 1,467,045
• Seed (cited) papers: 23,479
• ESI fields: 21
• Representative WoS categories: 21
• Q1 journals: 105
• Citation intent labels: 31 observed (30 valid; 7 canonical)
• Supplementary knowledge graph: 3,418,433 nodes / 6,855,117 edges
• Data snapshot: collected November 2025

IDCite preserves the scale, imbalance, and disciplinary heterogeneity of real-world scholarly citations across the life sciences, medicine, engineering, physical sciences, social sciences, humanities, and multidisciplinary domains.

code/
├── Data Collection/
│   ├── collect_by_journal.py
│   ├── paper_title.py
│   └── batch_paper_title_multi.py
│
├── IDCite Construction/
│   └── ontology.py
│
└── Technical Validation/
    └── idcite_technical_validation.py

README.md

IDCite is constructed by integrating multiple large-scale scholarly data sources:

Bibliographic metadata are collected via the Scopus API (using pybliometrics), providing journal articles, citation counts, and rich publication metadata. These records are used to identify influential seed papers based on journal-stratified citation statistics.

WoS subject categories are used to group journals by scientific field and to select Top-5 Q1 journals per representative category (105 journals across 21 categories), enabling journal-stratified, field-aware sampling.

The OpenAlex API is used for DOI resolution and large-scale citation-link retrieval (i.e., identifying papers that cite the selected seed papers).

The Semantic Scholar Graph API is used to retrieve citation context spans associated with each citing paper. Citation contexts correspond to textual spans surrounding in-text citation markers.

IDCite is built through a transparent and reproducible pipeline:

1. Seed paper acquisition & journal-stratified sampling

   • Journals grouped by WoS subject categories (21 categories × 5 journals)
   • Top 5% most-cited papers retained independently within each journal
   • Produces 23,479 multidisciplinary seed papers

2. Bibliographic metadata collection

   • Metadata harmonized across Scopus, OpenAlex, and Semantic Scholar

3. Citation-event extraction

   • Each citing publication is linked to a referenced seed paper as a directed citation event, preserving citation contexts and provenance

4. Semantic citation intent annotation

   • Citation events are annotated with citation intents using a graph neural network-based citation intent classification framework (weak supervision)

5. Entity normalization & DOI processing

   • Normalized identifiers for authors, affiliations, journals, cities, countries, fields, and citation intents

6. Supplementary ontology-ready knowledge graph construction

   • Heterogeneous nodes and typed edges (3,418,433 nodes / 6,855,117 edges), provided as a supplementary resource

7. Release

   • Structured Apache Parquet tables for citation events, publication metadata, normalized entities, and the knowledge graph

• Uses the Scopus API to collect journal-level bibliographic metadata.
• Implements article collection per journal, citation-count retrieval, and journal-stratified Top-5% cited paper selection.
• Produces intermediate artifacts used to identify influential seed papers.

• Core citation context extraction engine.
• Resolves DOIs (via OpenAlex if necessary) and retrieves citation links via the OpenAlex API.
• Retrieves citation context spans and intent signals via the Semantic Scholar Graph API.
• Outputs structured citation context records.

• Batch execution wrapper for paper_title.py.
• Iterates over lists of influential papers for large-scale context extraction across multiple journal groups.

• Builds the ontology-ready IDCite resource from the raw seed-paper and citing-context archives.
• Generates citation-event records, citing-paper and seed-paper tables, enriched citation events, and normalized entity lookup tables (authors, affiliations, journals, cities, countries, fields, intents).
• Constructs the ontology-ready knowledge graph (kg_nodes.parquet, kg_edges.parquet) with typed nodes and edges.
• Run with:

  python "code/IDCite Construction/ontology.py" --base-dir /path/to/wos_data

• Outputs are written to <base-dir>/citationhub_v1_ontology_ready/.

• Reproduces the technical-validation analyses reported in the paper: metadata completeness, citation-event referential integrity, citation-intent distribution and coverage, knowledge-graph integrity, and entity-table uniqueness.
• Writes per-check CSV tables and a bundled ZIP of all validation results.
• Run with:

  python "code/Technical Validation/idcite_technical_validation.py" \
      --data-dir /path/to/wos_data/citationhub_v1_ontology_ready --make-figures

The released IDCite resource is distributed as structured Parquet files:

Citation intents are generated by an automated graph neural network-based citation-intent classification framework and released as weak semantic supervision. The dataset contains 31 observed intent labels (30 valid categories plus one missing label retained for reproducibility), comprising seven canonical intents and additional composite categories. Composite labels are decomposed into their constituent intents, so a single citation event can contribute to multiple intent classes.

In addition to the core citation tables, IDCite provides an optional, ontology-ready knowledge graph (kg_nodes.parquet, kg_edges.parquet) as a supplementary resource for downstream graph analytics, network analysis, and entity-relationship exploration. It contains 3,418,433 nodes and 6,855,117 edges with no duplicate node identifiers and 100% source-node linkage.

• Node types: SeedPaper, CitingPaper, CitationEvent, Intent, Journal, Author, Affiliation, City, Country, Field
• Edge types: citation links (CitationEvent → CitingPaper / SeedPaper), intent assignment, authorship, affiliation, publication venue, geographic associations, and disciplinary assignments

IDCite supports a wide range of research scenarios, including:

• Citation-aware information retrieval
• Intent-aware ranking and re-ranking
• Citation recommendation and candidate generation
• Large-scale citation intent classification
• Scientometric and bibliometric analysis
• Interdisciplinary knowledge discovery
• Knowledge graph analytics and link prediction
• AI-assisted scientific discovery workflows

• Python 3.9+
• pandas, pyarrow, matplotlib (figures only), and pybliometrics (Scopus collection only)

Reproducing the full pipeline requires:

• Access to the Scopus API (institutional entitlement may be required)
• Access to the OpenAlex API (publicly available)
• Access to the Semantic Scholar Graph API (publicly available; rate limits apply)

API keys, where required, must be supplied via environment variables and are not included in this repository. Because the underlying scholarly infrastructures are continuously updated, IDCite should be interpreted as a snapshot of the scholarly ecosystem corresponding to the November 2025 collection period.

The software resources associated with IDCite are provided through two complementary repositories:

• Dataset construction pipeline (MDCite, this repository): https://github.com/kecau/MDCite
• Interactive dashboard and visualization platform (CitationHub): https://github.com/kecau/CitationHub

• Original MDContextCite database (Zenodo): https://zenodo.org/records/18536895
• IDCite processed dataset and graph database (Hugging Face): https://huggingface.co/datasets/Daniel0315/IDCite