LLM Usecase
LLM Usecase
Abstract
The electric vehicle (EV) battery supply
chain’s vulnerability to disruptions necessi-
tates advanced predictive analytics. We present
arXiv:2408.05357v2 [cs.AI] 21 Oct 2024
To address these challenges, we introduce SHIELD butions include: (1) an LLM-expert integration
(Schema-based Hierarchical Induction for EV sup- methodology for accurate, interpretable predic-
ply chain Disruption), a two-stage framework that tions; (2) a schema learning and news evaluation
integrates LLMs and domain expertise for predic- dataset spanning the EV battery lifecycle; (3) an
tive analytics in EV battery supply chains (Fig. 1): interactive schema curation system; and (4) ad-
vanced analytical techniques for supply chain anal-
1. Schema Learning (Sec. 3): We leverage LLMs
ysis. SHIELD offers a promising approach in sup-
(GPT-4o, Llama3-3b, Llama3-70b) to con-
ply chain risk management, addressing evolving
struct a comprehensive schema library—a
challenges across the EV industry and beyond.
structured representation of supply chain com-
ponents and their relationships—from diverse
2 Related Work
sources. An interactive system integrates ex-
pert knowledge, distilling supply chain ex- Supply Chain Risk Management. AI has been
pertise from specialized documents. This ap- increasingly applied to predict and manage supply
proach ensures analyses align with domain chain risks (Ganesh and Kalpana, 2022). Agent-
knowledge, capturing EV battery supply chain based approaches (Pino et al., 2010; Giannakis
complexities for accurate, interpretable predic- and Louis, 2011, 2016; Blos et al., 2015) facili-
tions that adapt to industry dynamics through tate inter-agent communication for forecasting but
continuous refinement. often suffer from limited predictive power and pa-
2. Disruption Analysis (Sec. 4): Building on our rameter constraints. Rule-based methods (Gal-
schema learning, we develop a comprehen- lab et al., 2019; Behret et al., 2012; Paul, 2015;
sive disruption prediction system. This sys- Paul et al., 2017; Awasthi et al., 2018; Camarillo
tem integrates fine-tuned RoBERTa (Liu et al., et al., 2018) offer decision frameworks with mini-
2019) for event detection, multi-dimensional mal quantitative insights. Machine Learning (ML)
similarity for matching, and Graph Convo- and Deep Learning (DL) techniques have improved
lutional Networks (GCNs) with logical con- forecasting and disruption prediction (Silva et al.,
straints for impact analysis. The resulting end- 2017; Hegde and Rokseth, 2020; Garvey et al.,
to-end system enables precise event extraction 2015; Ruz et al., 2020; Aljohani, 2023; Carbonneau
and reliable predictions in complex supply et al., 2008), yet many focus on predictive perfor-
chains, mitigating LLM hallucination risks mance at the expense of interpretability (Hendrik-
while maintaining efficiency. This approach sen, 2023; Makridis et al., 2023). Recent work has
offers a scalable solution for real-time supply explored large language models (LLMs) in supply
chain risk assessment and mitigation. chain management (Ray, 2023), but interpretabil-
ity remains a challenge. Our approach integrates
Evaluated on 12,070 paragraphs from 365 sources LLMs to enhance both predictive accuracy and in-
(2022-2023) (Sec. 5), SHIELD outperforms base- terpretability by extracting hierarchical knowledge-
line GCNs and LLM+prompt methods (e.g., GPT- graph structures to forecast disruptions.
4o) in disruption prediction. By integrating LLM
capabilities with domain expertise, this framework Schema Induction & Learning. Building on early
enhances supply chain risk assessment. Key contri- schema induction work (Anderson et al., 1979;
Evans, 1967), LLMs (Brown et al., 2020; Rae et al., the EV battery lifecycle, enabling our methods to
2021) have shown strong schema-learning abili- acquire expert knowledge for accurate, real-world
ties with minimal supervision. Recent strategies, predictions. More details are in Appx. B.1.
such as contextual explanations (Wei et al., 2021;
Schema Generation & Merging. Building upon
Lampinen et al., 2022), rationale-augmented mod-
our collected dataset, our Schema Learning Sys-
els (Wang et al., 2022b), and incremental prompt-
tem facilitates the extraction, visualization, and
ing (Li et al., 2023), have further refined schema
management of schemas from the 125 diverse tex-
induction. Transformer-based methods (Li et al.,
tual sources (Fig. 2). The process begins with data
2020, 2021) excel at schema representation through
cleaning using regular expressions and a locally
graph structures, with human feedback playing a
deployed Llama3-8b model. Subsequently, we em-
vital role in improving model accuracy (Mondal
ploy GPT-4o, Llama3-3b, and Llama3-70b with
et al., 2023; Yang et al., 2024; Zhang et al., 2023).
specific prompts to extract hierarchical structures
Our method leverages these advancements, com-
(H) capturing main events (E) and sub-events
bining human feedback with LLM-driven schema
(Esub ). More details are in Appx. C.
induction to improve accuracy and relevance in
disruption prediction. The extracted structures are then converted into in-
dividual schemas (Si ) and visualized as graphs,
Event Extraction & Analysis. Event extraction
demonstrating the hierarchical nature of the
has evolved from handcrafted features (Ahn, 2006)
schemas and the relationships between main events
to neural models like recurrent (Nguyen et al.,
and sub-events. These schemas are then integrated
2016; Sha et al., 2018), convolutional (Chen et al.,
into a singleSlibrary (Sfinal ), aggregating contexts
2015), graph (Zhang and Ji, 2021), and transformer- n
based networks (Liu et al., 2020). Advances in Snfinal = i=1 Ci ), merging events (Efinal =
(C
argument extraction (Wang et al., 2019), zero-shot i=1 Ei ), S
and updating event IDs for relations
(Rfinal = ni=1 Ri ). The detailed schema genera-
learning (Huang et al., 2018), and weak supervi-
tion and merging algorithm is provided in Appx. D.
sion (Chen et al., 2015) have boosted performance.
Our approach enhances event extraction by using To ensure efficient retrieval and updates, a dedi-
fine-tuned RoBERTa models and graph convolu- cated Database & Storage module manages schema
tional networks (GCNs) to capture complex event storage, while the Schema Management System in-
relationships and cascading effects, offering deeper corporates a Schema Viewer, Editor, collaboration
insights into supply chain disruptions compared to tools, and AI-driven suggestions are built to man-
traditional methods. age and annotate schemas (Appx. E). This human-
in-the-loop curated framework streamlines schema
3 Schema Learning for Supply Chain extraction and management, enabling interactive
Disruptions knowledge extraction from structured documents,
Schema Learning Dataset. Our dataset comprises leveraging supply chain experts’ insights.
239 diverse sources: 200 academic papers, 22 in- 4 Dynamic Analysis of Supply Chain
dustry reports, and 17 Wikipedia entries (Fig. 5 and
Disruptions
Fig. 6). This collection provides an up-to-date view
of the EV battery supply chain, covering advanced Supply Chain News Dataset. We developed an EV
battery technologies (e.g. LFP, NiMH), produc- Supply Chain News Dataset (January 2022 - De-
tion processes, and six key raw materials. We cat- cember 2023) to evaluate our system’s real-world
egorized events into 8 categories, three with long- performance (Appx. B.2). The dataset comprises
term impacts, subdivided into 18 subcategories. 247 articles from major news outlets and 118 en-
Our analysis includes five-year price trends for terprise reports from EV battery-related companies
all materials, correlated with 39 significant sup- (Fig. 7 and Fig. 8). After preprocessing—including
ply chain events. Industry expert feedback refined text extraction, language standardization, and noise
our categorization into 11 main categories with reduction—we obtained Meta data with 3,022 para-
27 subcategories, each illustrated with 1-2 real- graphs. We then fused international news with con-
world events (Tab. 6). The academic dataset was temporaneous corporate stories in the meta data,
distilled from 239 sources to 125 highly relevant creating 354 diverse documents comprising 12,070
entries. This dataset of over 1,000 events spans paragraphs. The final dataset contains approxi-
Figure 3: Overview of the supply chain disruption prediction pipeline, illustrating the integration of GCN-based
predictions, constrained prediction refinement, and argument coreference resolution.
mately 660K words (Table 9), providing a robust and score each event’s impact as:
foundation for evaluating supply chain disruption ImpactScore(ei ) = Centrality(ei )+Magnitude(ei )
detection and analysis. Comprehensive replication (5)
details, including the full dataset and preprocessing This scoring mechanism balances two crucial fac-
pipeline, are provided to ensure reproducibility. tors. Centrality(ei ) represents the event’s impor-
Event Extraction. Our pipeline extracts multi- tance within the network, reflecting its centrality or
faceted events from textual data, focusing on their influence in the supply chain context. Meanwhile,
impact on the EV battery supply chain. We begin Magnitude(ei ) quantifies the event’s impact inten-
with custom-trained SpaCy models2 for tokeniza- sity, indicating its severity or significance.
tion, sentence segmentation, named entity recogni- Finally, we apply logical constraints and argument
tion, and dependency parsing (Appx. F). coreference to ensure robustness:
Building on this, we deploy a fine-tuned RoBERTa LogicCoref(PC ) → PF (6)
model for cross-sentence event detection: producing a refined, logically consistent set of
event parameters PF . More implementation details
EventDetectmulti-sentence (T) → EC (1) are in Appx. F.
where T represents the input text and EC the de-
Event Matching & Instantiation. We link ex-
tected events. These events are then enriched with
tracted events with schema library to detect supply
contextual information using BERT:
chain disruption patterns using a multi-dimensional
approach of semantic and structural similari-
BERTcontext (EC ) → CE (2)
ties. We align each extracted event Eext ∈ Eext (ex-
generating contextual embeddings CE . To enhance tracted events) with each schema event Eschema ∈
analytical coherence, we implement coreference Eschema (schema events) using a composite similar-
resolution and event linking: ity:
Sim(Eext , Eschema ) = α · SemSim(Eext , Eschema )
CorefLink(EC ) → EL (3)
+ β · StrSim(Eext , Eschema )
This critical step, yielding linked events EL , main- (7)
tains contextual continuity across documents. Sub- where SemSim captures contextual meaning using
sequently, Conditional Random Fields (CRFs) ex- BERT embeddings, and StrSim assesses structural
tract event parameters PC : similarity. Specifically, semantic similarity mea-
sures contextual alignment using cosine similarity
CRF(EL ) → PC (4)
between BERT embeddings:
vext · vschema
Leveraging Graph Convolutional Networks SemSim(Eext , Eschema ) = (8)
(GCNs), we model complex event relationships ∥vext ∥∥vschema ∥
where vext and vschema are BERT embeddings of
2
https://spacy.io/models extracted and schema events. Similarly, structural
similarity evaluates parameter overlap using Jac- Disruption Prediction. Building on the extracted
card similarity: and matched events, we employ Graph Convolu-
tional Networks (GCNs), logical constraints, and
|Pext ∩ Pschema |
StrSim(Eext , Eschema ) = (9) argument coreference resolution to predict supply
|Pext ∪ Pschema | chain disruptions. Note that the events are repre-
where Pext and Pschema are the parameter sets for sented as nodes and interactions as edges using
the extracted and schema events. GCNs with the propagation rule:
Following the calculation of semantic and struc-
tural similarities, we refine matching using heuris- H(l+1) = σ(AH(l) W(l) ) (12)
tic rules from annotated datasets. Successful
where H(l) is the hidden state at layer l, A is the ad-
matches lead to event instantiation, enriching the
jacency matrix, W(l) is the weight matrix, and σ is
event representation with schema attributes:
a non-linear activation function. We optimize using
Instantiate(Ematched , Sschema ) → Einst (10) mean squared error loss with L2 regularization:
where Ematched represents the matched event,
N
Sschema yields the schema library, and Einst refers 1 X
L= (yi − ŷi )2 + λ∥W∥2 (13)
to the instantiated event with enriched attributes. N
i=1
To ensure logical adherence to schema constraints, where yi and ŷi are actual and predicted disruption
we perform consistency checks. These checks vali- scores, and λ is a regularization parameter. This
date the instantiated events against the schema li- approach balances prediction accuracy and model
brary, ensuring they conform to predefined logical complexity, preventing overfitting.
and structural constraints:
To ensure consistency with domain knowledge, we
ConsistencyCheck(Einst , Sschema ) (11) apply logical constraints, refining initial predictions
This step is crucial for maintaining the integrity of (ŷ) to produce final predictions (ŷ ′ ) that adhere to
the schema and the reliability of the predictions. known rules:
Finally, we incorporate a continuous improvement
ŷ ′ = arg min Constrain(ŷ)
process through manual review and feedback. Feed- ŷ ′ ∈Y (14)
back from domain experts is used to update and subject to C(ŷ ′ ) = true
refine the models, ensuring they adapt to new pat-
terns and maintain high performance. The complete where C represents constraint sets. For example, a
process is summarized in Algorithm 3. More im- constraint might ensure that a major supplier’s dis-
plementation details are in Appx. G. ruption increases risk for dependent manufacturers.
Algorithm 1 Supply Chain Disruption Prediction To further enhance the model’s contextual under-
1: Input: Historical supply chain events E, adjacency matrix standing, we incorporate argument coreference:
A, initial predictions ŷ
2: Output: Refined predictions ŷ ′
3: GCN-based Prediction ▷ Initial prediction using GCN Rij = arg, max; Coref(Ei , Ej )
4: for l = 1 to L do Ei ,Ej ∈E (15)
5: H(l+1) = σ(AH(l) W(l) ) ▷ Refer to Eq. 12
6: end for subject to Coref(Ei , Ej ) = true
7: ŷ ← H(L)
8: Constrained Prediction ▷ Apply logical constraints where (Ei , Ej ) denotes each event pair and Rij rep-
9: for each prediction ŷi do
10: ŷi′ ← Constrain(ŷi ) resents their relation. This AllenNLP-based model
11: such that C(ŷi′ ) = true ▷ Refer to Eq. 14 links entities across event mentions, recognizing
12: end for
13: Coreference Resolution ▷ Link related events
when different descriptions refer to the same inci-
14: for each pair of events (Ei , Ej ) do dent, thereby improving prediction accuracy and
15: Rij ← Coref(Ei , Ej ) ▷ Refer to Eq. 15 context comprehension. Algorithm 1 outlines our
16: if Rij is coreferential then
17: Link Ei and Ej
approach, combining GCN-based predictions, log-
18: end if ical constraints, and argument coreference resolu-
19: end for tion. Detailed examples and implementation guide-
20: Return: Refined predictions ŷ
lines are provided in Appx. H.
Table 1: Performance comparison of different LLMs on schema learning in stage 1.
ChatGPT4o Llama3-3b Llama3-70b
Individual Schemas Integrated Library Individual Schemas Integrated Library Individual Schemas Integrated Library
Precision 0.637 0.184 0.198 0.018 0.353 0.019
Recall 0.695 0.336 0.047 0.014 0.133 0.022
F-score 0.652 0.238 0.068 0.016 0.175 0.020
Table 2: Subjective evaluation by domain experts. erage F-score 0.687 vs 0.683 in 2022) indicates
Model Consistency Accuracy Completeness potential refinement in our model’s ability to adapt
GPT-4o 4.5 4.3 4.6 to evolving supply chain dynamics.
Llama3-3b 1.8 1.5 1.9
Llama3-70b 3.0 2.7 3.1 Disruption Detection. Our advanced GCNs model,
augmented with logical constraints and corefer-
ence resolution, was rigorously evaluated against
5 Experiments ablation versions and LLM+prompt methods. Ta-
Our evaluation comprises two parts: (1) Schema bles 4 and 5 present the comparative performance
Learning Assessment and (2) Supply Chain Disrup- metrics. The full system achieved the highest F-
tion Prediction. We assess learned schemas against score (0.732), significantly outperforming both ab-
expert knowledge and evaluate our schema induc- lation versions (GCNs+Logical Constraints: 0.707,
tion process’s effectiveness in predicting supply GCNs only: 0.685) and LLM+prompt methods
chain events. Detailed experimental setup and eval- (GPT-4o: 0.624). However, the incremental im-
uation metrics are in Appx. I. provement from the GCNs-only model to our full
system (0.685 to 0.732) suggests that while the
5.1 Schema Learning Performance additional components significantly enhance per-
formance, there remains substantial potential for
We evaluate GPT-4o, Llama3-3b, and Llama3-70b
further optimization in the future.
for schema learning, comparing individual schema
extraction and integrated library generation. Ta-
5.3 Qualitative Analysis & User Interface
bles 1 and 2 present quantitative metrics and sub-
jective evaluations by domain experts. GPT-4o out- Our qualitative analysis of SHIELD’s disruption
performs Llama models, achieving F-scores of predictions, focusing on real-world case studies
0.652 and 0.238 for individual schemas and in- (detailed in Appx. J), complements the quantitative
tegrated library generation, respectively. All mod- findings and further illuminates the system’s practi-
els perform better in individual schema extraction cal utility. A particularly salient example emerged
than integrated library generation, indicating chal- in SHIELD’s accurate prediction of a semiconduc-
lenges in schema integration. Subjective assess- tor shortage resulting from geopolitical tensions,
ments align with quantitative metrics, with GPT- made three weeks prior to widespread reporting.
4o scoring highest across all criteria (consistency: This early insight enabled proactive adjustments
4.5, accuracy: 4.3, completeness: 4.6). Individual to procurement strategies, thereby demonstrating
schemas show strong consistency and complete- the system’s considerable potential in mitigating
ness but slightly lower accuracy, suggesting a trade- complex supply chain risks. We have developed
off between comprehensive coverage and precise an interactive user interface (Fig. 4) for online dis-
detail representation. ruption analysis. This interface allows users to up-
load news report texts (Fig. 4a), evaluate prediction
5.2 Disruption Detection Performance scores, and edit visualization results for the final
Event Extraction & Matching. Table 3 presents disruption analysis (Fig. 4b). More details can be
quarterly results for 2022 and 2023 on event ex- found in Appx. K.
traction and matching using a supply chain news
dataset. Our system maintains consistent perfor- 5.4 Disruption Prediction Case Studies
mance across quarters, with F-scores ranging from Our system effectively forecasted key supply chain
0.671 to 0.700. This stability suggests robust gener- disruptions, providing insights that enabled stake-
alization across different time periods and varying holders to take proactive actions. For example, fol-
event types. The slight improvement in 2023 (av- lowing the passage of the Inflation Reduction Act
(a) User interface for inputting news reports. (b) Visualization and editing of final prediction results.
Figure 4: User interface for online disruption analysis in stage 2, showing the process from news report input to the
visualization and editing of prediction results. More examples are in Appx. K.
Table 3: Event extraction and matching in supply chain news dataset.
Year 2022 2023
Quarter Q1 (Jan-Mar) Q2 (Apr-Jun) Q3 (Jul-Sep) Q4 (Oct-Dec) Q1 (Jan-Mar) Q2 (Apr-Jun) Q3 (Jul-Sep) Q4 (Oct-Dec)
Precision 0.714 0.692 0.705 0.689 0.712 0.698 0.703 0.690
Recall 0.675 0.662 0.678 0.655 0.688 0.670 0.681 0.657
F-score 0.694 0.677 0.691 0.671 0.700 0.684 0.692 0.673
Table 5: Performance comparison of direct human inter- decision-making in a volatile global market.
action with LLMs on disruption prediction.
Model Precision Recall F-score 6 Conclusion
GPT-4o 0.641 0.608 0.624
We present SHIELD, a two-stage framework that
Llama3-3b 0.522 0.489 0.505 integrates Large Language Models (LLMs) with
Llama3-70b 0.557 0.523 0.540 domain expertise, yielding promising results in EV
Our Method 0.754 0.712 0.732
battery supply chain analytics and risk assessment.
While demonstrating particular strength in early
(2022), which incentivized domestic EV battery disruption detection and event prediction for criti-
production, our system predicted potential material cal battery materials, significant challenges remain
shortages. By analyzing shifts in global material in schema integration, real-time adaptability, and
flows and the effects of policy changes on supply- error reduction. Future research will systematically
demand dynamics, it enabled early interventions address these limitations, enhance the system’s ro-
to minimize risks. Similarly, during geopolitical bustness, and explore broader applications across
tensions between Australia and China in 2023, our diverse industries and supply chain ecosystems.
system identified vulnerabilities in the lithium sup-
Acknowledgments
ply chain by monitoring export data and geopoliti-
cal developments, helping stakeholders adapt their This research was supported by the Carnegie Mel-
sourcing strategies in time. In another instance, lon Manufacturing Futures Institute and the Man-
the system anticipated cobalt supply issues result- ufacturing PA Innovation Program. The authors
ing from labor strikes and regulatory changes in also thank the School of Computer Science (SCS)
the Democratic Republic of Congo (2023), allow- at Carnegie Mellon University, particularly the
ing companies to diversify sources and increase High Performance Computing (HPC), for provid-
inventory buffers. These cases, detailed further in ing essential computational resources. The views
Appendix J, illustrate how data-driven predictions expressed are those of the authors and do not nec-
enhance supply chain resilience and support timely essarily reflect those of the funding agencies.
References Rotem Dror, Haoyu Wang, and Dan Roth. 2022. Zero-
shot on-the-fly event schema induction. arXiv preprint
David Ahn. 2006. The stages of event extraction. In arXiv:2210.06254.
Proceedings of the Workshop on Annotating and Rea-
soning about Time and Events, pages 1–8. Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei
Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben
Abeer Aljohani. 2023. Predictive analytics and machine Zhou, et al. 2022. Resin-11: Schema-guided event pre-
learning for real-time supply chain risk mitigation and diction for 11 newsworthy scenarios. In Proceedings of
agility. Sustainability, 15(20):15088. the 2022 Conference of the North American Chapter of
John R Anderson, Paul J Kline, and Charles M the Association for Computational Linguistics: Human
Beasley Jr. 1979. A general learning theory and its Language Technologies: System Demonstrations, pages
application to schema abstraction1. In Psychology of 54–63.
learning and motivation, volume 13, pages 277–318.
Selby H Evans. 1967. A brief statement of schema
Elsevier.
theory. Psychonomic Science, 8(2):87–88.
Anjali Awasthi, Kannan Govindan, and Stefan Gold.
2018. Multi-tier sustainable global supplier selection Maryam Gallab, Hafida Bouloiz, Youssef Lamrani
using a fuzzy ahp-vikor based approach. International Alaoui, and Mohamed Tkiouat. 2019. Risk assessment
Journal of Production Economics, 195:106–117. of maintenance activities using fuzzy logic. Procedia
computer science, 148:226–235.
Hülya Behret, Başar Öztayşi, and Cengiz Kahraman.
2012. A fuzzy inference system for supply chain risk A Deiva Ganesh and P Kalpana. 2022. Future of ar-
management. In Practical Applications of Intelligent tificial intelligence and its influence on supply chain
Systems: Proceedings of the Sixth International Confer- risk management–a systematic review. Computers &
ence on Intelligent Systems and Knowledge Engineer- Industrial Engineering, 169:108206.
ing, Shanghai, China, Dec 2011 (ISKE2011), pages Myles D Garvey, Steven Carnovale, and Sengun
429–438. Springer. Yeniyurt. 2015. An analytical framework for supply net-
Maurício F Blos, Robson M Da Silva, and Paulo E work risk propagation: A bayesian network approach.
Miyagi. 2015. Application of an agent-based sup- European Journal of Operational Research, 243(2):618–
ply chain to mitigate supply chain disruptions. IFAC- 627.
PapersOnLine, 48(3):640–645.
Mihalis Giannakis and Michalis Louis. 2011. A multi-
Tom Brown, Benjamin Mann, Nick Ryder, Melanie agent based framework for supply chain risk manage-
Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind ment. Journal of Purchasing and Supply Management,
Neelakantan, Pranav Shyam, Girish Sastry, Amanda 17(1):23–31.
Askell, et al. 2020. Language models are few-shot
learners. Advances in neural information processing Mihalis Giannakis and Michalis Louis. 2016. A multi-
systems, 33:1877–1901. agent based system with big data processing for en-
hanced supply chain agility. Journal of Enterprise In-
Shu Cai and Kevin Knight. 2013. Smatch: an evaluation formation Management, 29(5):706–727.
metric for semantic feature structures. In Proceedings
of the 51st Annual Meeting of the Association for Com- Jeevith Hegde and Børge Rokseth. 2020. Applica-
putational Linguistics (Volume 2: Short Papers), pages tions of machine learning methods for engineering risk
748–752. assessment–a review. Safety science, 122:104492.
Alvaro Camarillo, José Ríos, and Klaus-Dieter Althoff. Christian Hendriksen. 2023. Artificial intelligence for
2018. Knowledge-based multi-agent system for manu- supply chain management: Disruptive innovation or
facturing problem solving process in production plants. innovative disruption? Journal of Supply Chain Man-
Journal of manufacturing systems, 47:115–127. agement, 59(3):65–76.
Real Carbonneau, Kevin Laframboise, and Rustam Vahi- Lifu Huang, Heng Ji, Kyunghyun Cho, Ido Dagan, Se-
dov. 2008. Application of machine learning techniques bastian Riedel, and Clare R Voss. 2018. Zero-shot trans-
for supply chain demand forecasting. European journal fer learning for event extraction. In 56th Annual Meet-
of operational research, 184(3):1140–1154. ing of the Association for Computational Linguistics,
ACL 2018, pages 2160–2170. Association for Computa-
Angel X Chang and Christopher D Manning. 2012. Su- tional Linguistics (ACL).
time: A library for recognizing and normalizing time
expressions. In Lrec, volume 12, pages 3735–3740. Andrew K Lampinen, Ishita Dasgupta, Stephanie CY
Chan, Kory Matthewson, Michael Henry Tessler, Anto-
Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and nia Creswell, James L McClelland, Jane X Wang, and
Jun Zhao. 2015. Event extraction via dynamic multi- Felix Hill. 2022. Can language models learn from expla-
pooling convolutional neural networks. In Proceedings nations in context? arXiv preprint arXiv:2204.02329.
of the 53rd Annual Meeting of the Association for Com-
putational Linguistics and the 7th International Joint Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang,
Conference on Natural Language Processing (Volume Kyunghyun Cho, Heng Ji, Jiawei Han, and Clare Voss.
1: Long Papers), pages 167–176. 2021. The future is not one-dimensional: Complex
event schema induction by graph modeling for event the Seventeenth Conference on Computational Natural
prediction. arXiv preprint arXiv:2104.06344. Language Learning, pages 143–152.
Manling Li, Qi Zeng, Ying Lin, Kyunghyun Cho, Heng Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Mil-
Ji, Jonathan May, Nathanael Chambers, and Clare Voss. lican, Jordan Hoffmann, Francis Song, John Aslanides,
2020. Connecting the dots: Event graph schema induc- Sarah Henderson, Roman Ring, Susannah Young, et al.
tion with path language modeling. In Proceedings of 2021. Scaling language models: Methods, analy-
the 2020 Conference on Empirical Methods in Natural sis & insights from training gopher. arXiv preprint
Language Processing (EMNLP), pages 684–695. arXiv:2112.11446.
Sha Li, Ruining Zhao, Manling Li, Heng Ji, Chris Partha Pratim Ray. 2023. Leveraging deep learning
Callison-Burch, and Jiawei Han. 2023. Open- and language models in revolutionizing water resource
domain hierarchical event schema induction by incre- management, research, and policy making: A case for
mental prompting and verification. arXiv preprint chatgpt. ACS ES&T Water, 3(8):1984–1986.
arXiv:2307.01972.
Gonzalo A Ruz, Pablo A Henríquez, and Aldo Mas-
Jian Liu, Yubo Chen, Kang Liu, Wei Bi, and Xiaojiang careño. 2020. Sentiment analysis of twitter data during
Liu. 2020. Event extraction as machine reading com- critical events through bayesian networks classifiers.
prehension. In Proceedings of the 2020 conference Future Generation Computer Systems, 106:92–104.
on empirical methods in natural language processing
(EMNLP), pages 1641–1651. Alexander Schrijver. 1998. Theory of linear and integer
programming. John Wiley & Sons.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man-
dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Lei Sha, Feng Qian, Baobao Chang, and Zhifang Sui.
Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A 2018. Jointly extracting event triggers and arguments
robustly optimized bert pretraining approach. arXiv by dependency-bridge rnn and tensor-based argument
preprint arXiv:1907.11692. interaction. In Proceedings of the AAAI conference on
artificial intelligence, volume 32.
Georgios Makridis, Philip Mavrepis, and Dimosthenis
Kyriazis. 2023. A deep learning approach using natural Xiaoming Shi, Siqiao Xue, Kangrui Wang, Fan Zhou,
language processing and time-series forecasting towards James Zhang, Jun Zhou, Chenhao Tan, and Hongyuan
enhanced food safety. Machine Learning, 112(4):1287– Mei. 2024. Language models can improve event pre-
1313. diction by few-shot abductive reasoning. Advances in
Neural Information Processing Systems, 36.
Ishani Mondal, Michelle Yuan, Aparna Garimella, Fran-
cis Ferraro, Andrew Blair-Stanek, Benjamin Van Durme, Nathalie Silva, Luís Miguel DF Ferreira, Cristóvão
Jordan Boyd-Graber, et al. 2023. Interactiveie: To- Silva, Vanessa Magalhães, and Pedro Neto. 2017. Im-
wards assessing the strength of human-ai collaboration proving supply chain visibility with artificial neural net-
in improving the performance of information extraction. works. Procedia Manufacturing, 11:2083–2090.
arXiv preprint arXiv:2305.14659.
Jannik Strötgen and Michael Gertz. 2013. Multilin-
Thien Huu Nguyen, Kyunghyun Cho, and Ralph Grish- gual and cross-domain temporal tagging. Language
man. 2016. Joint event extraction via recurrent neural Resources and Evaluation, 47:269–298.
networks. In Proceedings of the 2016 conference of the
North American chapter of the association for computa- Naushad UzZaman, Hector Llorens, James Allen,
tional linguistics: human language technologies, pages Leon Derczynski, Marc Verhagen, and James Puste-
300–309. jovsky. 2012. Tempeval-3: Evaluating events, time
expressions, and temporal relations. arXiv preprint
Sanjoy Kumar Paul. 2015. Supplier selection for man- arXiv:1206.5333.
aging supply risks in supply chain: a fuzzy approach.
The International Journal of Advanced Manufacturing Hongwei Wang, Zixuan Zhang, Sha Li, Jiawei Han,
Technology, 79:657–664. Yizhou Sun, Hanghang Tong, Joseph P Olive, and Heng
Ji. 2022a. Schema-guided event graph completion.
Sanjoy Kumar Paul, Ruhul Sarker, and Daryl Essam. arXiv preprint arXiv:2206.02921.
2017. A quantitative model for disruption mitigation
in a supply chain. European Journal of Operational Sijia Wang, Mo Yu, Shiyu Chang, Lichao Sun, and Lifu
Research, 257(3):881–895. Huang. 2021. Query and extract: Refining event extrac-
tion as type-oriented binary decoding. arXiv preprint
Raúl Pino, Isabel Fernández, David de la Fuente, José arXiv:2110.07476.
Parreño, and Paolo Priore. 2010. Supply chain mod-
elling using a multi-agent system. Journal of Advances Xiaozhi Wang, Ziqi Wang, Xu Han, Zhiyuan Liu, Juanzi
in Management Research, 7(2):149–162. Li, Peng Li, Maosong Sun, Jie Zhou, and Xiang Ren.
2019. Hmeae: Hierarchical modular event argument
Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, extraction. In Proceedings of the 2019 Conference on
Hwee Tou Ng, Anders Björkelund, Olga Uryupina, empirical methods in natural language processing and
Yuchen Zhang, and Zhi Zhong. 2013. Towards robust the 9th international joint conference on natural lan-
linguistic analysis using ontonotes. In Proceedings of guage processing (EMNLP-IJCNLP), pages 5777–5783.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc
Le, Ed Chi, and Denny Zhou. 2022b. Rationale-
augmented ensembles in language models. arXiv
preprint arXiv:2207.00747.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin
Guu, Adams Wei Yu, Brian Lester, Nan Du, An-
drew M Dai, and Quoc V Le. 2021. Finetuned lan-
guage models are zero-shot learners. arXiv preprint
arXiv:2109.01652.
Diyi Yang, Sherry Tongshuang Wu, and Marti A Hearst.
2024. Human-ai interaction in the age of llms. In Pro-
ceedings of the 2024 Conference of the North American
Chapter of the Association for Computational Linguis-
tics: Human Language Technologies (Volume 5: Tuto-
rial Abstracts), pages 34–38.
Tianyi Zhang, Isaac Tham, Zhaoyi Hou, Jiaxuan Ren,
Liyang Zhou, Hainiu Xu, Li Zhang, Lara J Martin,
Rotem Dror, Sha Li, et al. 2023. Human-in-the-loop
schema induction. arXiv preprint arXiv:2302.13048.
Zixuan Zhang and Heng Ji. 2021. Abstract meaning
representation guided graph encoding and decoding for
joint information extraction. In Proc. The 2021 Confer-
ence of the North American Chapter of the Association
for Computational Linguistics-Human Language Tech-
nologies (NAACL-HLT2021).
A Extended Related Work approach leverages these advancements by employ-
ing an LLM-driven framework that integrates hu-
Supply Chain Risk Management. AI techniques
man feedback and expert knowledge into a human-
have been increasingly applied to predict and mit-
in-loop system, thereby enhancing the practical
igate supply chain risks (Ganesh and Kalpana,
accuracy and relevance of induced schemas.
2022). While agent-based approaches (Pino et al.,
2010; Giannakis and Louis, 2011, 2016; Blos et al., Event Extraction & Analysis. Event extraction
2015) enable inter-agent communication for fore- has evolved from manually crafted features (Ahn,
casting, they often lack robust predictive capabil- 2006) to neural models, including recurrent net-
ities and have limited parameter sets. Rule-based works (Nguyen et al., 2016; Sha et al., 2018), con-
reasoning methods (Gallab et al., 2019; Behret volutional networks (Chen et al., 2015), graph net-
et al., 2012; Paul, 2015; Paul et al., 2017; Awasthi works (Zhang and Ji, 2021), and transformers (Liu
et al., 2018; Camarillo et al., 2018) offer decision- et al., 2020). Recent research has focused on event
making frameworks but provide minimal quantita- argument extraction (Wang et al., 2019) and ex-
tive insights. To address these limitations, Machine plored zero-shot learning (Huang et al., 2018) and
Learning (DL) and Deep Learning (DL) techniques weak supervision (Chen et al., 2015) to enhance
have been employed (Silva et al., 2017; Hegde and performance. Our approach incorporates various
Rokseth, 2020; Garvey et al., 2015; Ruz et al., event extraction techniques, utilizing fine-tuned
2020; Aljohani, 2023; Carbonneau et al., 2008), RoBERTa models and graph convolutional net-
enhancing demand forecasting and disruption pre- works (GCNs) to capture and analyze complex
diction (Hendriksen, 2023; Makridis et al., 2023). event relationships and their cascading impacts.
Recent studies have begun exploring the potential This approach enables a deeper understanding of
of large language models (LLMs) in supply chain supply chain disruptions, distinguishing our system
management (Ray, 2023). However, most current from traditional extraction techniques.
works prioritize predictive performance over inter-
pretability, hindering practitioners’ ability to make B Dataset
informed decisions. Our approach addresses this
gap by integrating LLMs for schema induction, ex- B.1 Schema Learning Dataset
tracting hierarchical knowledge-graph structures Our research began by examining the current state
from academic resources to predict supply chain of EV batteries, focusing on the predominant types
disruptions, thereby enhancing both predictive per- in use, such as lithium iron phosphate and nickel
formance and interpretability. lithium batteries. We analyzed the battery produc-
Schema Induction & Learning. Building on foun- tion process and identified key raw materials, in-
dational works (Anderson et al., 1979; Evans, cluding lithium, cobalt, nickel, and graphite. Sub-
1967), recent advancements in language model- sequently, we investigated the primary sources and
ing have revolutionized schema induction. Large- production volumes of these materials. Through an
scale language models (LLMs) (Brown et al., 2020; extensive review of literature and statistical data,
Rae et al., 2021) have demonstrated remarkable we categorized significant supply chain events into
capabilities in learning and generating schemas eight groups, three of which have long-term im-
with minimal supervision. Researchers have ex- pacts. Each category was further divided into sub-
plored various strategies to enhance these mod- categories, and real-world events were identified to
els, including contextual explanations (Wei et al., illustrate their impact on raw material supplies.
2021; Lampinen et al., 2022), rationale-augmented We also analyzed price trends for key raw mate-
ensembles (Wang et al., 2022b), and incremental rials over the past five years, using data from the
prompting (Li et al., 2023). Transformer-based ap- London Metal Exchange (LME)3 , to assess how
proaches (Li et al., 2020, 2021) have proven partic- news events influenced these prices. This research
ularly effective in managing schema generation for produced an initial scenario document listing the
complex scenarios, representing schemas as graphs. primary raw materials for EV batteries, their price
Integrating human feedback (Mondal et al., 2023; trends, and an analysis of events causing supply
Yang et al., 2024; Zhang et al., 2023) has been chain issues and price fluctuations. Each category
crucial in refining schema induction processes, ad-
3
dressing the limitations of automated methods. Our https://www.lme.com/en/
included at least one real-world example to demon- 2023. We initially developed a Python crawler us-
strate its impact. ing the requests and BeautifulSoup packages
to scrape news titles and summaries from multiple
The initial document was then submitted for re- websites, such as Google News4 and Infoplease5 .
view by a supply chain expert. Based on the ex- This resulted in a collection of 643 records. To filter
pert’s feedback, we refined the events affecting the out news unrelated to the supply chain, we designed
EV battery supply chain into 11 main categories, a prompt leveraging GPT-4o’s language capabili-
three with long-term impacts, and subdivided them ties. Using the summaries from the list, GPT-4o
into 27 subcategories. Each subcategory was il- helped categorize events into various types, such as
lustrated with 1-2 real-world events, and raw ma- natural disasters, wars, trade policy, and political
terials were further subdivided, such as different issues, tagging the relevant countries and regions.
grades of nickel and types of lithium. Categories
with minimal impact were removed, resulting in a Subsequently, we employed large language models
comprehensive and refined scenario document. (LLMs) to evaluate the relevance of each news
event to the EV battery supply chain based on the
Based on the scenario document, we identified following criteria, each worth 25 points:
the raw materials and events related to the EV
battery supply chain and began collecting an aca- 1. Whether natural disasters or humanitarian
demic document dataset. Our data sources included crises occurred in raw material production
Wikipedia entries, supply chain-related papers, and areas, such as China, Australia, Indonesia,
industrial reports. After obtaining the raw data, Congo, Chile, Canada, or in EV production
we manually removed redundant information and countries, such as China, Japan, South Korea,
noise, retaining only the paragraphs most relevant and the United States.
to the EV battery supply chain. Through meticu-
lous organization, we compiled an academic doc- 2. Whether the event could affect trade relations
ument dataset consisting of 125 entries, distilled in the aforementioned countries, including
from 239 diverse sources: 200 academic papers, trade issues, sanctions, or wars.
22 industry reports, and 17 Wikipedia entries. This 3. Whether the event could potentially disrupt
curated dataset provides a focused knowledge base international shipping routes due to conflicts
essential for analyzing and understanding the com- or natural disasters near these routes.
plexities of the EV battery supply chain.
4. Whether the event is directly related to inter-
The resulting dataset encompasses a wealth of national trade.
knowledge related to the EV battery supply chain,
covering aspects such as raw material procurement, Events scoring below 25 points were initially elim-
manufacturing processes, supply chain logistics, inated, followed by a manual review of the remain-
and market dynamics. Table 6 presents the event ing events, resulting in a refined list of 247 sup-
categories and example events. Events marked with ply chain-related news events. The text data was
* indicate potential long-term impacts, highlighting sourced from reputable media outlets, including
the various types of disruptions and their implica- Reuters6 , BBC7 , and CNN8 . Additionally, to gather
tions for the supply chain. Fig. 5 and 6 illustrate the contemporaneous supply chain status information,
sources of the academic papers, Wikipedia entries, we scraped company news and analysis reports
and industry reports used in compiling the dataset, from EV battery-related companies like Ford, Volk-
demonstrating the breadth and diversity of our data swagen, and CATL, as well as supply chain-related
sources. By synthesizing this information, we aim websites, totaling 118 reports. The raw data, in-
to provide a robust foundation for understanding cluding titles, publication dates, and content, was
the complexities and challenges associated with the organized chronologically.
EV battery supply chain.
The raw data contained invalid information and
B.2 Supply Chain News Dataset 4
https://news.google.com/
5
https://www.infoplease.com/
To comprehensively test our system, we con- 6
https://www.reuters.com/
structed an EV Supply Chain News Dataset cov- 7
https://www.bbc.com/
8
ering the period from January 2022 to December https://edition.cnn.com/
Table 6: Event Categories and Example Events. Events marked with * indicate potential long-term impacts.
Event Category Subcategory Example
Investment from U.S. or Ally U.S. invests in EV battery industry in Canada
Acquisition and Investment∗
Investment from Other Country China invests in cobalt mines in DRC
Demand Change Demand for ore from the Philippines increases
Changes in Supply and Demand
Supply Change Tight supplies of nickel ore in Indonesia
Production Halt or Reduction Katanga halts cobalt mining
Enterprise Issue Enterprise Crisis Katanga faces an equity crisis
Production Plan Adjustment Kellyton Graphite increases production by 15%
Macroeconomy The U.S. and EU face continued inflation
Economic Environment
Competition and Market Structure Competition from China’s low-priced EVs
Product Upgrading CATL releases Kirin battery
EV Battery Technology Progress∗
Production Technology Progress Development of graphene batteries
Forced Labor Forced labor in production
Humanitarian and Ethical Crisis Use of Child Labor Child labor in cobalt mining in DRC
Human Rights Issue Large numbers of refugees enter Europe
Production Affected by Disaster Australia floods affect lithium mining
Natural Disaster
Transportation Affected by Disaster Tsunami destroys ports, disrupts shipping
Regional Tension Tensions between North and South Korea
∗ Changes in International Relations China’s relations with the West deteriorate
Political Issue
Industry Nationalization Nationalization of the lithium industry in Chile
Government Intervention Europe promotes EVs for environmental reasons
Sign a Supply Agreement Sign a Supply Agreement PE signs EV battery supply agreement with Tesla
Export Controls China restricts graphite exports
Trade Policy Tax and Duties China’s tax rebates to EV companies
Trade Barriers US tariffs on Chinese EV batteries
Internal Disorder or Rebellion Civil unrest in Yemen
War and Conflict War Between Nations Russo-Ukrainian War
Geopolitical Crisis Houthi rebels attack merchant ship
advertisements, which were cleaned using regular the extensive scope of the collected data. Fig. 7
expressions to remove most invalid information. presents the categories and examples of news ar-
We deployed Llama3-8b to filter out embedded ad- ticles in the dataset, while Fig. 8 shows the distri-
vertisements, ensuring the dataset’s purity and accu- bution of sources in the news dataset, emphasizing
racy. After cleaning, irrelevant content was reduced the dataset’s diversity and comprehensiveness. The
by 15%, and all data was systematically stored in a dataset covers global events that could impact the
database, resulting in a refined meta dataset of 365 supply chain, such as natural disasters, trade issues,
news documents. The metadata contains approxi- wars, enterprise issues, etc.
mately 152,000 words and 3,000 paragraphs. To
validate our system’s ability to detect connections C Hierarchical Structure Extraction
between events, we randomly merged international
news with contemporaneous corporate stories from We utilize large language models (LLMs) to extract
the same quarter, creating 354 fused documents hierarchical structures (H) that capture main events
for a more diverse and challenging dataset. The fi- (E) and sub-events (Esub ) based on our prompt, as
nal fusion dataset contains approximately 660,000 illustrated in Fig. 9.
words and 12,000 paragraphs. In a hierarchical structure (H):
Upon completing dataset collection, we conducted • An event (E) refers to anything that happens
preliminary statistics and analysis on the news related to the EV battery supply chain. There
dataset. Table 8 shows the number of event types can be multiple events ⟨E1 , E2 , . . . , En ⟩ in
included in each quarter in the news dataset, provid- one hierarchical structure H.
ing a comprehensive overview of the various events
tracked over time. Table 9 details the number of • An event_id is a unique identifier code as-
words and paragraphs in the dataset, highlighting signed for each specific event.
◼ The Electric Vehicle Supply Chain Ecosystem: Changing Roles of Automotive Suppliers
• Academic Paper ◼ Electric Vehicle Battery Supply Chain and Critical Materials: A Brief Survey of State of the Art
◼ Electric vehicle battery supply chain and critical materials: a brief survey of state of the art ◼ Estimating the environmental impacts of global lithium-ion battery supply chain: A temporal, geographical, and
◼ Challenges and recent developments in supply and value chains of electric vehicle batteries: A sustainability perspective technological perspective
◼ Cost-effective supply chain for electric vehicle battery remanufacturing ◼ A SWOT Analysis of the UK EV Battery Supply Chain
◼ The supply chain for electric vehicle batteries ◼ The Electric Vehicle Supply Chain Ecosystem: Changing Roles of Automotive Suppliers
◼ Critical issues in the supply chain of lithium for electric vehicle batteries ◼ Building a North American electric vehicle supply chain
◼ Building a Robust and Resilient U.S. Lithium Battery Supply Chain ◼ Friend-shoring battery supply chains
◼ At the mining or extraction stage, major risks include the location of the deposit, cost, geopolitical environment, and ◼ Estimating the environmental impacts of global lithium-ion battery supply chain: A temporal, geographical, and
mining regulations technological perspective
◼ A SWOT Analysis of the UK EV Battery Supply Chain ◼ Global Supply Chains of EV Batteries
◼ Lithium-ion battery supply chain: enabling national electric vehicle and renewables targets ◼ Building a Sustainable Electric Vehicle Battery Supply Chain
◼ Sustainable electric vehicle batteries for a sustainable world: perspectives on battery cathodes, environment, supply ◼ INVESTIGATING THE U.S. BATTERY SUPPLY CHAIN AND ITS IMPACT ON ELECTRIC VEHICLE COSTS THROUGH 2032
chain, manufacturing, life cycle, and policy ◼ GLOBAL STATE OF Sustainable ELECTRIC VEHICLE BATTERIES
◼ Developing pricing strategy to optimise total profits in an electric vehicle battery closed loop supply chain ◼ Collection and recycling decisions for electric vehicle end-of-life power batteries in the context of carbon emissions
◼ Optimising quantity of manufacturing and remanufacturing in an electric vehicle battery closed-loop supply chain reduction
◼ Life-cycle implications and supply chain logistics of electric vehicle battery recycling in California ◼ The Paradox of Green Growth: Challenges and Opportunities in Decarbonizing the Lithium-Ion Supply Chain
◼ Analyzing challenges for sustainable supply chain of electric vehicle batteries using a hybrid approach of Delphi and ◼ Implications of the Electric Vehicle Manufacturers’ Decision to Mass Adopt Lithium-Iron Phosphate Batteries
Best-Worst Method ◼ Reducing new mining for electric vehicle battery metals: responsible sourcing through demand reduction strategies and
◼ Determining requirements and challenges for a sustainable and circular electric vehicle battery supply chain: A mixed- recycling
methods approach ◼ Automated assembly of Li-ion vehicle batteries: A feasibility study
◼ Graphite resources, and their potential to support battery supply chains, in Africa ◼ Field Study and Multimethod Analysis of an EV Battery System Disassembly
◼ Lithium-ion battery supply chain considerations: analysis of potential bottlenecks in critical metals ◼ COVID-19 disrupts battery materials and manufacture supply chains, but outlook remains strong
◼ The case for recycling: Overview and challenges in the material supply chain for automotive li-ion batteries ◼ An overview of global power lithium-ion batteries and associated critical metal recycling
◼ Global Value Chains: Graphite in Lithium-ion Batteries for Electric Vehicles ◼ Does China's new energy vehicles supply chain stock market have risk spillovers? Evidence from raw material price
◼ Traceability methods for cobalt, lithium, and graphite production in battery supply chains effect on lithium batteries
◼ The global cycle of Graphite - A dynamic Material Flow Analysis (2020-2050) of the natural, synthetic and recycled ◼ Deep-sea nodules versus land ores: A comparative systems analysis of mining and processing wastes for battery-metal
graphite value chains to understand the supply of LIB anodes supply chains
◼ Natural graphite demand and supply—Implications for electric vehicle battery requirements ◼ A game theoretic approach for analyzing electric and gasoline-based vehicles’ competition in a supply chain under
◼ Mateerial System Analysis of five battery-related raw materials: Cobalt, Lithium, Manganese, Natural Graphite, Nickel government sustainable strategies: A case study of South Korea
◼ Life cycle assessment of natural graphite production for lithium-ion battery anodes based on industrial primary data ◼ China's lithium supply chain: Security dynamics and policy countermeasures
◼ Sustainability challenges throughout the electric vehicle battery value chain ◼ Exploring recycling options in battery supply chains – a life cycle sustainability assessment
◼ Electric vehicle battery chemistry affects supply chain disruption vulnerabilities ◼ Lithium-Ion Battery Recycling in the Circular Economy: A Review
◼ Global competition in the lithium-ion battery supply chain: a novel perspective for criticality analysis ◼ Implications of circular production and consumption of electric vehicle batteries on resource sustainability: A system
◼ Supply risks of lithium-ion battery materials: An entire supply chain estimation dynamics perspective
◼ Vulnerable links in the lithium-ion battery supply chain ◼ On the influence of second use, future battery technologies, and battery lifetime on the maximum recycled content of
◼ The global battery arms race: lithium-ion battery gigafactories and their supply chain future electric vehicle batteries in Europe
◼ Towards the lithium-ion battery production network: Thinking beyond mineral supply chains ◼ Assessing batteries supply chain networks for low impact vehicles
◼ Challenges and opportunities in lithium-ion battery supply ◼ Mapping a circular business opportunity in electric vehicle battery value chain: A multi-stakeholder framework to create
◼ Identifying supply risks by mapping the cobalt supply chain a win–win–win situation
◼ The Cobalt Supply Chain and Environmental Life Cycle Impacts of Lithium-Ion Battery Energy Storage Systems ◼ An applied analysis of the recyclability of electric vehicle battery packs
◼ Environmental sustainability and supply resilience of cobalt ◼ A Novel Prediction Process of the Remaining Useful Life of Electric Vehicle Battery Using Real-World Data
◼ Perspectives on cobalt supply through 2030 in the face of changing demand ◼ Capturing the battery value-chain opportunity
◼ An integrated supply chain analysis for cobalt and rare earth elements under global electrification and constrained ◼ Digital Twin-Driven Framework for EV Batteries in Automobile Manufacturing
resources ◼ Electric vehicle battery state changes and reverse logistics considerations
◼ Sources of uncertainty in the closed-loop supply chain of lithium-ion batteries for electric vehicles ◼ Optimising the geospatial configuration of a future lithium ion battery recycling industry in the transition to electric
◼ Battery technology and recycling alone will not save the electric mobility transition from future cobalt shortages vehicles and a circular economy☆
◼ Global value chains: cobalt in lithium-ion batteries for electric vehicles ◼ Can Cobalt Be Eliminated from Lithium-Ion Batteries?
◼ Global electrification of vehicles and intertwined material supply chains of cobalt, copper and nickel ◼ Sizing and Locating Planning of EV Centralized-Battery-Charging-Station Considering Battery Logistics System
◼ The cobalt supply chain and life cycle assessment of lithium-ion battery energy storage systems ◼ Taming the Hydra: Funding the Lithium Ion Supply Chain in an Era of Unprecedented Volatility
◼ Towards the lithium-ion battery production network: Thinking beyond mineral supply chains ◼ Battery technology and recycling alone will not save the electric mobility transition from future cobalt shortages
◼ Vertically Integrated Supply Chain of Batteries, Electric Vehicles, and Charging Infrastructure: A Review of Three ◼ Resilience assessment of the lithium supply chain in China under impact of new energy vehicles and supply interruption
Milestone Projects from Theory of Constraints Perspective ◼ Optimal policy for the recycling of electric vehicle retired power batteries
◼ The behavioural evolution of the smart electric vehicle battery reverse supply chain under government supervision ◼ Data requirements and availabilities for a digital battery passport – A value chain actor perspective
◼ Managing resource dependencies in electric vehicle supply chains: a multi-tier case study ◼ Trade structure and risk transmission in the international automotive Li-ion batteries trade
◼ Application of sustainable supply chain finance in end-of-life electric vehicle battery management: a literature review ◼ The Supply Chain Diversification and India–South Korea Cooperation in a Contested East Asia in the Post-COVID-19 Era
◼ Structural characteristics and disruption ripple effect in a meso-level electric vehicle Lithium-ion battery supply chain ◼ Toward Sustainable Reuse of Retired Lithium-ion Batteries from Electric Vehicles
network ◼ Steering extended producer responsibility for electric vehicle batteries
◼ Battery global value chain and its technological challenges for electric vehicle mobility ◼ Traceability Management Strategy of the EV Power Battery Based on the Blockchain
◼ Building a competitive advantage for Indonesia in the development of the regional EV battery chain ◼ Electric vehicle lithium-ion battery recycled content standards for the US – targets, costs, and environmental impacts
◼ Recycling mode selection and carbon emission reduction decisions for a multi-channel closed-loop supply chain of ◼ What is the contribution of different business processes to material circularity at company-level? A case study for
electric vehicle power battery under cap-and-trade policy electric vehicle batteries
◼ A sustainable circular supply chain network design model for electric vehicle battery production using internet of things ◼ To shred or not to shred: A comparative techno-economic assessment of lithium ion battery hydrometallurgical
and big data recycling retaining value and improving circularity in LIB supply chains
◼ The CO2 Impact of the 2020s Battery Quality Lithium Hydroxide Supply Chain ◼ Value recovery from spent lithium-ion batteries: A review on technologies, environmental impacts, economics, and
◼ Electric vehicle supply chain management: A bibliometric and systematic review supply chain
◼ Long-term Indonesia's Nickel Supply Chain Strategy for Lithium-Ion Battery as Energy Storage System ◼ Optimal choice of power battery joint recycling strategy for electric vehicle manufacturers under a deposit-refund
◼ Battery Nickel Bottlenecks system
◼ Life-cycle analysis, by global region, of automotive lithium-ion nickel manganese cobalt batteries of varying nickel ◼ Collection mode choice of spent electric vehicle batteries: considering collection competition and third-party economies
content of scale
◼ Battery minerals from Finland: Improving the supply chain for the EU battery industry using a geometallurgical approach ◼ Electric vehicle battery capacity allocation and recycling with downstream competition
◼ Exploring recycling options in battery supply chains–a life cycle sustainability assessment ◼ Tackling EV Battery Chemistry in View of Raw Material Supply Shortfalls
◼ Assessment of social sustainability hotspots in the supply chain of lithium-ion batteries ◼ Which is better? Business models of partial and cross ownership in an NEV supply chain
◼ Conflict minerals and battery materials supply chains: A mapping review of responsible sourcing initiatives ◼ Materials availability and supply chain considerations for vanadium in grid-scale redox flow batteries
◼ Analysis of nickel sulphate datasets used in lithium-ion batteries ◼ Circularity of Lithium-Ion Battery Materials in Electric Vehicles
◼ Design of battery supply chains under consideration of environmental and socio-economic criteria ◼ Assessment of end-of-life electric vehicle batteries in China: Future scenarios and economic benefits
◼ Analysis of international nickel flow based on the industrial chain ◼ The Resilience of the Renewable Energy Electromobility Supply Chain: Review and Trends
◼ Value recovery from spent lithium-ion batteries: A review on technologies, environmental impacts, economics, and ◼ Industrial Policy, Trade, and Clean Energy Supply Chains
supply chain ◼ Concurrent design of product and supply chain architectures for modularity and flexibility: process, methods, and
◼ Dynamic evolution of the zinc-nickel battery industry and evidence from China application
◼ A perspective on the sustainability of cathode materials used in lithium‐ion batteries ◼ A sustainable framework for the second-life battery ecosystem based on blockchain
◼ An improved resource midpoint characterization method for supply risk of resources: integrated assessment of Li-ion ◼ Comparative evaluation and policy analysis for recycling retired EV batteries with different collection modes
batteries ◼ HOW TECHNOLOGY, RECYCLING, AND POLICY CAN MITIGATE SUPPLY RISKS TO THE LONG-TERM TRANSITION TO ZERO-
◼ Deep‐sea nodules versus land ores: A comparative systems analysis of mining and processing wastes for battery‐metal EMISSION VEHICLES
supply chains ◼ Decarbonizing the automotive sector: a primary raw material perspective on targets and timescales
◼ Industrial policy, trade, and clean energy supply chains ◼ The Emerging Electric Vehicle and Battery Industry in Indonesia: Actions around the Nickel Ore Export Ban and a SWOT
◼ The electric vehicle revolution: Critical material supply chains, trade and development Analysis
◼ Industrial policy for electric vehicle supply chains and the US-EU fight over the Inflation Reduction Act ◼ The Emerging Electric Vehicle and Battery Industry in Indonesia: Actions around the Nickel Ore Export Ban and a SWOT
◼ Strategic Battery Autarky: Reducing Foreign Dependence in the Electric Vehicle Supply Chain Analysis
◼ Electric vehicle battery secondary use under government subsidy: A closed-loop supply chain perspective ◼ Reverse Logistics Network Design of Electric Vehicle Batteries considering Recall Risk
◼ Blockchain review for battery supply chain monitoring and battery trading ◼ Battery capacity needed to power electric vehicles in India from 2020 to 2035
◼ Trade structure and risk transmission in the international automotive Li-ion batteries trade ◼ The future of the automotive sector: Emerging battery value chains in Europe
◼ The EV Revolution: Critical Material Supply Chains, Trade, and Development ◼ Radical innovations as supply chain disruptions? A paradox between change and stability
◼ An Overview of the Lithium Supply Chain ◼ End of Electric Vehicle Batteries: Reuse vs. Recycle
◼ Comparison of lithium-ion battery supply chains–a life cycle sustainability assessment ◼ Securing Decarbonized Road Transport – a Comparison of How EV Deployment Has Become a Critical Dimension of
◼ The ev transition: Key market and supply chain enablers Battery Security Strategies for China, the EU, and the US.
◼ The Lithium Supply Crunch Doesn't Have to Stall Electric Cars ◼ A Review on Battery Market Trends, Second-Life Reuse, and Recycling
◼ Hydrometallurgical Routes to Close the Loop of Electric Vehicle (EV) Lithium-Ion Batteries (LIBs) Value Chain: A Review ◼ Life cycle impact assessment of electric vehicle battery charging in European Union countries
◼ Global warming potential of lithium-ion battery cell production: Determining influential primary and secondary raw ◼ Assessing socio-economic risks in the supply chain of materials required for vehicle electrification
material supply routes ◼ Intelligent disassembly of electric-vehicle batteries: a forward-looking overview
◼ Lithium mining: How new production technologies could fuel the global EV revolution ◼ Research on decision optimization of new energy vehicle supply chain considering demand disruptions under dual credit
◼ The cobalt and lithium global supply chains: status, risks and recommendations policy
◼ Sustainable value chain of retired lithium-ion batteries for electric vehicles ◼ Transition to electric vehicles in China: Implications for private motorization rate and battery market
◼ Assessing the potential of quebec lithium industry: Mineral reserves, lithium-ion batteries production and greenhouse ◼ Mirroring in production? Early evidence from the scale-up of Battery Electric Vehicles (BEVs)
gas emissions ◼ Life-Cycle Assessment Considerations for Batteries and Battery Materials
◼ Current and Future Global Lithium Production Till 2025 ◼ Operation Management of Multiregion Battery Swapping–Charging Networks for Electrified Public Transportation
◼ Lithium-Ion Batteries Recycling Trends and Pathways: A Comparison Systems
◼ Alternative battery chemistries and diversifying clean energy supply chains ◼ Spatial modeling of a second-use strategy for electric vehicle batteries to improve disaster resilience and circular
◼ Lithium and cobalt economy
◼ A Study on the Cradle-to-Gate Environmental Impacts of Automotive Lithium-ion Batteries ◼ On the sustainability of lithium ion battery industry – A review and perspective
◼ Determining requirements and challenges for a sustainable and circular electric vehicle battery supply chain: A mixed- ◼ Comparison of Electric Vehicle Lithium-Ion Battery Recycling Allocation Methods
methods approach ◼ Manufacturing value chain for battery electric vehicles in Pakistan: An assessment of capabilities and transition
◼ Status and gap in rechargeable lithium battery supply chain: importance of quantitative failure analysis pathways
◼ Critical Factors to Consider in Purchasing for a Sustainable Inbound Supply Chain: A Perspective on Large Scale Lithium- ◼ The End of Globalized Production? Supply-Chain Resilience, Technological Sovereignty, and Enduring Global
ion Battery Manufacturing Interdependencies in the Post-Pandemic Era
◼ Assessing batteries supply chain networks for low impact vehicles ◼ Environmental feasibility of secondary use of electric vehicle lithium-ion batteries in communication base stations
◼ A comparative assessment of value chain criticality of lithium-ion battery cells ◼ Supply chain risks of critical metals: Sources, propagation, and responses
◼ Conflict minerals and battery materials supply chains: A mapping review of responsible sourcing initiatives ◼ Decentralized Planning of Lithium-Ion Battery Production and Recycling
◼ Battery Critical Materials Supply Chain Challenges and Opportunities: Results of the 2020 Request for Information (RFI) ◼ Improvements in electric vehicle battery technology influence vehicle lightweighting and material substitution decisions
and Workshop ◼ Rethinking Chinese supply resilience of critical metals in lithium-ion batteries
◼ Key Strategic Issues in Supply Chain Domain Pertaining to Battery Industry. ◼ Optimal pricing strategy in the closed-loop supply chain using game theory under government subsidy scenario: A case
◼ Identifying trends in battery technologies with regard to electric mobility: evidence from patenting activities along and study
across the battery value chain ◼ Predictive model for energy consumption of battery electric vehicle with consideration of self-uncertainty route factors
◼ Electric Vehicle Battery Supply Chain and Critical Materials: A Brief Survey of State of the Art ◼ Perspectives on Cobalt Supply through 2030 in the Face of Changing Demand
◼ Estimating the environmental impacts of global lithium-ion battery supply chain: A temporal, geographical, and ◼ McKinsey Electric Vehicle Index: Europe cushions a global plunge in EV sales
technological perspective ◼ Lithium in International Law: Trade, Investment, and the Pursuit of Supply Chain Justice
◼ A SWOT Analysis of the UK EV Battery Supply Chain
Table 7: The number of event types included in each quarter in the news dataset.
Event Type 2022Q1 2022Q2 2022Q3 2022Q4 2023Q1 2023Q2 2023Q3 2023Q4
Acquisition and Investment 0 5 4 0 2 2 2 1
Changes in Supply and Demand 6 5 4 3 3 1 2 3
Enterprise Issue 3 1 1 3 0 1 0 0
Economic Environment 3 3 5 6 4 2 6 1
Humanitarian and Ethical Crisis 1 5 3 1 2 0 2 3
Natural Disaster 6 8 6 5 6 6 6 6
Political Issue 3 14 18 16 14 10 15 10
EV Battery Technology Progress 1 1 1 2 4 3 2 0
Sign a Supply Agreement 0 3 1 1 2 2 1 4
Trade Policy 2 6 5 2 5 1 2 9
War and Conflict 3 6 12 10 5 7 7 7
Table 8: The number of times each country is mentioned in the news dataset in each quarter.
Country 2022Q1 2022Q2 2022Q3 2022Q4 2023Q1 2023Q2 2023Q3 2023Q4
USA 6 9 18 15 17 12 9 10
China 1 10 12 6 12 4 4 9
EU 3 11 6 4 8 4 8 2
Japan 0 1 0 1 1 2 2 1
Russia 4 10 4 14 6 6 4 3
Other 20 42 38 33 31 23 35 40
Table 9: Statistics of the number of words and para- • Use ’and’ if no sub-events can be missing.
graphs in the dataset.
• Use ’or’ if some sub-events can be missing.
Total Paragraphs Total Words
Meta Data 3,022 152,489 • Use ’xor’ if sub-events cannot exist simulta-
Fusion Data 12,070 660,054 neously.
Relations describe the connections between events.
• A description provides a detailed 2-3 sentence For example, if ev1.2 is caused by (happens after)
textual explanation of the event. ev1.1, it is expressed as ’ev1.1>ev1.2’.
• Participants include all sub-events (Esub ) re- Our prompt includes demonstration and chain of
lated to the main event, and a subsubevent thought (CoT) techniques:
(Esubsub ) can be used if an event is part of a • We manually annotated the hierarchical struc-
sub-event within the hierarchy. ture for one text in the schema learning dataset
to use as an example in the prompt.
The suffix P0.5 indicates the importance of a sub-
event to its parent event. The Gate specifies the • We provided a step-by-step CoT, showing how
relationship between the main event and its sub- E and Esub in H were extracted from specific
events: sentences in the schema learning dataset.
• Acquisition and Investment ◼ US battery supply chain investments reach US$92 billion since Biden took office
◼ Exclusive: Canada to invest C$2 billion on mineral strategy for EV battery supply chain ◼ Biden’s EV bet is a gamble on critical minerals
◼ China's EV battery materials industry set for $11bn capacity buildup ◼ Dead EV batteries turn to gold with US incentives
◼ Why carmakers are pouring billions into new electric vehicle battery factories ◼ DOE intends to award up to $37M to advance EV battery recycling, transportation and design
◼ Private equity in talks with UK's BMI for EV battery exposure ◼ UK Inflation Rate Reaches 40-Year High As Food Prices Surge
◼ CATL and Indonesia jointly build a nearly $6 billion power battery industry chain project ◼ Bank Of England To Get More Aggressive With 50 BPS Hike Later In The Week
◼ Durham battery storage company raises $100 million ◼ Huge Tax Cuts Are Being Questioned By Investors As Pounds Sinks
◼ Volkswagen announces $20 billion effort to build its own EV batteries ◼ Biden Threatens Windfall Tax As He Accuses Oil Companies Of War Profiteering
◼ Panasonic to open $4B EV battery plant in Kansas ◼ Inflation Of The United Kingdom Jumps To 41-Year High Of 11.1%
◼ CATL announces construction of second European factory in Hungary ◼ China Sets 5% As Their Economic Growth Target For 2023
◼ Tesla co-founder’s startup gets $2 billion to boost EV battery production ◼ Ukraine Secures First IMF Loan To A Country At War
◼ CATL and HGP establish partnership to jointly promote 5GWh battery energy storage application ◼ Largest Oil Refinery In Africa Launched By Aliko Dangote
◼ GM and POSCO Future M Investing $1bn in North American EV Battery Supply Chain ◼ China’s Economy Experienced A Growth Of 6.3% In The Second Quarter
◼ Companies invest in EV battery factories in Europe ◼ China To Kickstart Economy, After Plans To Improve Internal Migration
◼ Ecoprobm, SK, Ford investing in québec; building cathode plant to solidify EV supply chain in NA ◼ A 40% Windfall Tax Was Approved By Italian Government As A Result Of Soaring Profits
◼ Redwood Materials raises $1B to expand circular battery supply chain in US ◼ France Plans To End The Use Of Fossil Fuels By 2030
◼ Nissan leads $2.5 billion investment to build two more EVs in UK ◼ South Africa Gets $1 Billion Loan From World Bank To Tackle Power Crisis
Subevent N.1
subevent: [Subevent Name]
event_id: evN.1
description: [Detailed Description]
participants: [Subsubevent 1] evN.1.1_P[Importance], ...
Gate: [Gate]
Relations: [Event Relations]
Subsubevent N.1.1
subsubevent: [Subsubevent Name]
event_id: evN.1.1
description: [Detailed Description]
participants: [Subsubsubevent 1] evN.1.1.1_P[Importance], ...
Gate: [Gate]
Relations: [Event Relations]
‘‘‘
The prompt given to the LLMs is detailed and spe- the EV battery supply chain domain.
cific, ensuring that the models understand the ex-
act format and type of information we are extract- D Schema Generation & Merging
ing. By integrating demonstration and CoT tech-
niques, our prompt provides clear guidance to the With human-in-the-loop schema induction, our
LLMs, improving the accuracy and relevance of schema learning dataset generated 125 individ-
the extracted structures. Below is an example of ual schemas ⟨S1 , S2 , . . . , S125 ⟩. To create a single
the prompt used in Fig. 9. comprehensive schema, it is essential to merge all
individual schemas into a final schema (Sfinal ). The
To validate our approach, we tested the prompt with process of merging schema format files involves
various texts from the schema learning dataset. The systematically integrating multiple schemas into
hierarchical structures extracted were compared a cohesive schema. The key components include
with manually annotated structures to ensure accu- context, id, events, and relations. These compo-
racy and consistency. This process ensured that the nents determine the information in each event and
LLMs reliably produced high-quality hierarchical its correlation with other events, hence the merging
structures that aligned with expert knowledge in process must address all of them.
Use the provided example for guidance:
### Example:
**Input Paragraph**:
‘‘‘
Three main methods are used in lithium-ion recycling: pyrometallurgical, hydrometallurgical,
bioleaching, and direct recycling. The battery is melted in a hot furnace to recover some of the
cathode metal in pyrometallurgy. Pyrometallurgy employs extreme heat to transform metal oxides
into cobalt, copper, iron, and nickel alloys. Although it has a straightforward process and a
reasonably mature technology, the main drawbacks are its high cost and high environmental
pollution. Hydrometallurgy is a metal recovery method involving aqueous solutions to perform
leaching processes to precipitate a particular metal. In hydrometallurgy, specialized solution
reagents are primarily used to leach the targeted metals out from the cathode substance.
Although it is a highly effective and power-efficient method, its drawbacks include a lengthy
production time and a complicated process. Combinations of both pyrometallurgy and
hydrometallurgy are also used due to their advantages in sorting starting materials for cells.
The bioleaching technique uses bacteria to retrieve precious metals, but it is challenging
because the bacteria need a substantial amount of time to grow and are easily susceptible to
contamination.
‘‘‘
‘‘‘
Event 1
event: lithium-ion recycling
event_id: ev1
description: Methods for recycling lithium-ion batteries including pyrometallurgical,
hydrometallurgical, bioleaching, and direct recycling.
participants: pyrometallurgical ev1.1_P1, hydrometallurgical ev1.2_P1, bioleaching ev1.3_P1
Gate: or
Relations: ev1.1>ev1.3, ev1.2>ev1.3
Subevent 1.1
subevent: pyrometallurgical
event_id: ev1.1
description: Employs extreme heat to transform metal oxides into cobalt, copper, iron, and nickel
alloys.
participants: metal oxides ev1.1.1_P1, cobalt ev1.1.2_P0.5, copper ev1.1.3_P0.5, iron ev1.1.4_P0.5,
nickel alloys ev1.1.5_P0.5
Gate: and
Relations: ev1.1.1>ev1.1.2, ev1.1.1>ev1.1.3, ev1.1.1>ev1.1.4, ev1.1.1>ev1.1.5
Subevent 1.2
subevent: hydrometallurgy
event_id: ev1.2
description: Uses aqueous solutions to leach targeted metals out from the cathode substance.
participants: xxxx
Gate: xxxx
Relations: xxxx
Subevent 1.3
subevent: bioleaching
event_id: ev1.3
description: Uses bacteria to retrieve precious metals.
participants: xxxx
Gate: xxxx
Relations: xxxx
‘‘‘
In this stage, we utilize a trained graph neural net- In this phase, we utilize coreference entity links
work specifically designed for schema graphs to and instantiated entities to generate predictions for
score and select unmatched events in the instanti- the arguments associated with the predicted events.
ated graph. Key steps include: Key steps include:
• Graph Neural Network: A GCN is trained • Coreference Links: Coreference entity links
on schema graphs to learn representations of specified in the schema are used to ensure
nodes and edges. The propagation rule is consistency among entity mentions:
given by: Rij = arg, max Coref(Ei , Ej )
(l+1) (l) (l) Ei ,Ej ∈E (28)
H = σ(AH W ) (26)
subject to Coref(Ei , Ej ) = true
where H(l) is the hidden state at layer l, A is
the adjacency matrix, W(l) is the weight ma- where (Ei , Ej ) denotes each event pair and
trix, and σ is a non-linear activation function. Rij represents their relation.
• Instantiated Entities: Instantiated entities strates a logical hierarchical structure, while the
from the previous stages are leveraged to gen- schema produced from news reports presents a
erate arguments for predicted events. well-defined temporal sequence. Experts manu-
ally review the schemas to verify these attributes,
• Final Output: This stage produces the final
providing qualitative feedback on the logical co-
prediction output, including both events and
herence and comprehensiveness of the extracted
their arguments.
structures. Each schema is rated on a scale from 1
to 5, where 1 indicates poor quality and 5 indicates
I Experiment Details
excellent quality.
I.1 Experiment Setup
Objective Disruption Detection. We compare the
In Experiment 5.1, we evaluate the efficacy of three instantiated schemas learned by our system with
distinct Large Language Models (LLMs) in ex- manually annotated ground truth to assess the de-
tracting hierarchical structures from our schema gree of overlap. This comparison uses an evalu-
learning dataset. Leveraging domain expert knowl- ation metric similar to Smatch (Cai and Knight,
edge, we annotate individual schemas for each 2013), which involves breaking down both our
article in our academic corpus using our propri- schema and the ground truth into quadruples of the
etary system viewer and editor. We then employ form relation(event1, event2, importance). For in-
the methodology outlined in Appx. D to synthesize stance, the event of Raw Material Mining includes
these schemas into an integrated library. This com- the subevent of Lithium Mining with an associ-
bination of individual schemas and the integrated ated importance value, represented by the quadru-
library serves as the ground truth for our hierar- ple subevent(raw material mining, lithium mining,
chical information extraction phase. Our schema importance). Other relations include participants,
learning performance evaluation consists of two gates, sequential events, etc.
key components. First, we compare the hierarchical
information extracted by the three LLMs against To evaluate the results, we 1) map the events in
our established ground truth. Second, we assess the learned schema Sl to those in the ground-truth
the consistency, accuracy, and completeness of the schema Sgt , 2) establish a one-to-one mapping of
hierarchical structures derived from the textual con- quadruples between the learned schema Sl and the
tent of each article in the schema learning dataset, ground-truth schema Sgt , 3) calculate Precision,
with domain experts actively participating in this Recall, and F-score as follows:
evaluation process. number of matched quadruples in Sl
Precision =
In Experiment 5.2, we apply a similar annotation total quadruples in Sl
(29)
methodology to our news dataset as used for the
schema learning dataset. However, annotating the
news dataset presents unique challenges, as news number of matched quadruples in Sl
reports typically do not explicitly elucidate the con- Recall =
total quadruples in Sgt
nections between events. Instead, they often em- (30)
ploy speculative language to describe event interre-
lations. To ensure annotation accuracy, we heavily
Precision · Recall
rely on domain knowledge derived from scenario F-score = 2 · (31)
Precision + Recall
documents throughout the annotation process. Sub-
sequently, we utilize the ground truth extracted J Disruption Prediction Case Studies
from these reports to evaluate our system’s perfor-
mance in predicting news report outcomes. J.1 Case 1: Impact of the Inflation Reduction
Act in August 2022
I.2 Evaluation Metrics In August 2022, the United States passed the Infla-
Subjective Schema Learning. For subjective tion Reduction Act, which included significant in-
schema evaluation, we ensure that the event centives for domestic EV battery production. This
schemas generated from each paper and news re- led to a rapid increase in investments but also high-
port are consistent, accurate, and complete. The lighted potential material shortages, causing disrup-
schema derived from academic papers demon- tions in the EV battery supply chain.
System Prediction: Our system predicted the pos- analyzing news reports on strike activities and up-
sibility of short-term material shortages by analyz- dates on government regulations in the DRC. It
ing the market response data to the Inflation Re- also assessed historical data on cobalt supply and
duction Act, monitoring global distribution reports demand to identify vulnerabilities and integrated
of EV battery materials, and assessing the impact expert feedback on the impact of labor strikes and
of increased domestic production incentives on the regulatory changes on cobalt production.
supply and demand balance. Outcome: The system provided early warnings
Outcome: The system identified the risks posed by about the potential disruptions, enabling compa-
the sudden increase in demand for battery materials, nies to adjust their supply chain strategies. This
providing early warnings to stakeholders. This included diversifying sources of cobalt and increas-
allowed them to take proactive measures such as ing inventories to buffer against supply shortages.
securing long-term supply contracts and exploring
alternative materials to mitigate potential shortages. K SHIELD’s User Interface
J.2 Case 2: Lithium Supply Chain Disruption The SHIELD user interface is designed to be in-
in Early 2023 tuitive and user-friendly, facilitating the efficient
upload and analysis of news reports.
In early 2023, significant disruptions in the lithium
supply chain were caused by escalating geopolitical News Report Upload. On the right side of the in-
tensions between Australia and China. As Australia terface, users can upload their collected news report
is one of the world’s largest suppliers of lithium, texts. It includes a text box for input and a submis-
political factors heavily influenced its export poli- sion button to upload the report (see Fig. 11a). Key
cies, severely impacting the global supply chain for features include:
EV batteries, which rely heavily on lithium. • Upload Box: Allows users to paste or type
System Prediction: Our system accurately pre- their news report texts.
dicted the potential supply disruption by analyzing • Submit Button: Initiates the analysis process
various news reports on geopolitical developments once the report is uploaded.
and export data. The system monitored news re-
• Uploaded Reports List: Displays previously
lated to geopolitical tensions between Australia and
uploaded news reports, enabling users to re-
China, analyzed export data indicating changes in
view and compare past submissions easily.
Australia’s lithium export policies, and integrated
insights from scenario documents highlighting the Disruption Analysis Results. After submitting a
dependence of the EV battery supply chain on Aus- news report, users can view the real-time results of
tralian lithium exports. the disruption analysis on the left side of the inter-
Outcome: The system flagged the risk of Aus- face (see Fig. 11b). The comprehensive overview
tralia’s export restrictions to China, providing early of the analysis include:
warnings of potential disruptions in the EV bat- • Generated Schema: Displays the hierarchi-
tery supply chain. This allowed stakeholders to cal of events identified in the news report.
proactively seek alternative sources and mitigate
• Events List: Lists all detected events and
the impact on production.
their details, allowing to see which events
J.3 Case 3: Nickel and Cobalt Supply Issues were identified and how they are connected.
in March 2023 • Evaluation Score: Shows the real-time eval-
uation score, assessed against the schema li-
In March 2023, a major disruption in the global
brary for accuracy and completeness.
supply chain occurred due to large-scale worker
strikes and regulatory changes in the Democratic • Schema Editing: Allows to edit the gener-
Republic of Congo (DRC), a primary supplier of ated schema. Users can make changes to the
cobalt. Cobalt is crucial for EV batteries, and the structure, relationships, and details of events.
disruption had a significant negative impact on the • Regenerate Evaluation: Users can choose to
global supply chain. regenerate the evaluation score based on the
System Prediction: Our system successfully fore- edited schema, ensuring that the modifications
casted the potential supply chain interruptions by are reflected in the updated score.
L Author Contributions Frontend Interface:
Schema Learning Dataset: • Aike Shi and Yifei Dong: Provided the design.
• Yuzhi Hu: Created the scenario document. • Wei Liu: Optimized the design.
• Yifei Dong: Collected paper lists and Wiki- • Aike Shi: Implemented the interface.
data lists. Paper:
• Aike Shi: Collected Wikipedia lists and • Yifei Dong: Contributed to schema learning
weekly report lists. parts.
• Yifei Dong, Wei Liu, and Aike Shi: Extracted • Yuzhi Hu: Contributed to the news dataset
paragraphs from collected articles. and schema dataset parts and created and opti-
mized Figs. 5, 6, 7, and 8.
Supply Chain News Dataset:
• Wei Liu, Yifei Dong, and Aike Shi: Con-
• Yuzhi Hu: Conducted data crawling and clas- tributed to Figs. 1, 2, 3, 4, 9, 10, and 11.
sification.
– Yifei Dong and Wei Liu designed all fig-
• Yuzhi Hu, Yifei Dong, Wei Liu, and Aike Shi: ures.
Labeled ground truth for the news dataset.
– Wei Liu created and optimized all fig-
Schema Learning: ures.
• Yifei Dong and Aike Shi: Designed and mod- – Aike Shi designed the user interface ele-
ified prompts for generating structured infor- ments for Figs. 4, 10, and 11.
mation, designed the format for structured in-
• Jason O’Connor: Contributed to paper revi-
formation, wrote scripts for converting struc-
sions and suggestions.
tured information to SDF, and for generating
structured information using Llama3. • Zhi-Qi Cheng: Organized and rewrote the en-
tire paper, and supervised the modification of
• Yifei Dong, Aike Shi, Wei Liu, and Yuzhi all figures.
Hu: Generated structured information using
GPT-4o with zero-shot learning. Presentation and Project Webpage:
Figure 11: User interface for the disruption prediction analysis in SHIELD.