A Mini Project Report

On

NEXT WORD PREDICTOR

Submitted in Partial fulfillment of requirements for the award of the
degree of

BACHELOR OF ENGINEERING

In

COMPUTER SCIENCE & ENGINEERING

By

S. DHANUSH 245522733053

K. RAMCHARAN 245522733030

M. PRAVEEN 245522733035

Mrs.D.Mounika
Assistant Professor, Department of CSE

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

KESHAV MEMORIAL ENGINEERING COLLEGE

Approved by AICTE, New Delhi & Affiliated to Osmania University, Hyderabad)
D.No. 10 TC-111, Kachavanisingaram (V), Ghatkesar (M), Medchal-Malkajgiri, Telangana – 500 088
Website:www.kmec.in, Email-id: principal@kmec.in, M: +91 40 29560274
August - 2025
 KESHAV MEMORIAL ENGINEERING COLLEGE
Approved by AICTE, New Delhi & Affiliated to Osmania University, Hyderabad)
D.No. 10 TC-111, Kachavanisingaram (V), Ghatkesar (M), Medchal-Malkajgiri, Telangana – 500 088

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

CERTIFICATE

This is to certify that this is a bonafide record of the project report titled “NEXT
WORD PREDICTOR” which is being presented as the Mini Project report by

1. S DHANUSH 245522733053

2. K RAMCHARAN 245522733030

3. M PRAVEEN 245522733035

In partial fulfillment for the award of the degree of Bachelor of Engineering in

Computer Science & Engineering affiliated to the Osmania University, Hyderabad.

Mrs.D.Mounika,Assistant Proffessor Mrs.D.Mounika, Assitant Professor

Guide of the Project Mini-Project Coordinator
Department of Computer Science & Engineering, Department of Computer Science & Engineering,
Keshav Memorial Engineering College, Keshav Memorial Engineering College,
Hyderabad. Hyderabad.

Head of the Department

Department of Computer Science and Engineering,
Keshav Memorial Engineering College,
Hyderabad.

Submitted for Viva-Voce Examination held on ____________________________

 Vision & Mission of KMEC

Vision of the College:

⮚ To be the fountain head in producing highly skilled, globally competent engineers.

Producing quality graduates trained in the latest software technologies and related
tools and striving to make India a world leader in software products and services.

Mission of the College:

⮚ To provide a learning environment that inculcates problem solving skills,

professional, ethical responsibilities, lifelong learning through multi modal platforms
and prepare students to become successful professionals.
⮚ To establish industry institute Interaction to make students ready for the industry.
⮚ To provide exposure to students on latest hardware and software tools.
⮚ To promote research-based projects/activities in the emerging areas of technology
convergence.
⮚ To encourage and enable students to not merely seek jobs from the industry but also to
create new enterprises.
⮚ To induce a spirit of nationalism which will enable the students to develop,
understand lndia's challenges and to encourage them to develop effective solutions.
⮚ To support the faculty to accelerate their learning curve to deliver excellent service to
students.
 Vision & Mission of CSE
Vision of the CSE
Producing quality graduates trained in the latest software technologies and
related tools and striving to make India a world leader in software products and services.
To achieve academic excellence by imparting in depth knowledge to the students,
facilitating research activities and catering to the fast growing and ever-changing
software industrial demands and societal needs

Mission of the CSE

 To create a faculty pool which has a deep understanding and passion for algorithmic
thought process, which can then impart these skills to the students. The faculty will be
chosen with extreme care and not merely based on qualification. The faculty will have
to demonstrate substantial programming and solution skill during the selection
process. The faculty will be nurtured to retain their cutting edge all through their
tenure.

 To impart skills in Computer Science which will not necessarily be part of the
university prescribed curriculum. These skills may include programming languages,
frameworks, platforms and communication skills which will help in the development
of a well-rounded Computer Science professional.

 To inculcate an ability in students to pursue Computer Science education throughout

their lifetime. To exchange the use of multimodal learning platform including e-
learning, blended learning and also remote testing and skilling.

 To ensure that the student’s scope of knowledge is not limited to the conventional and
traditional use of Computer Science. The student will be given exposure to different
domains, different paradigms and exposed to the financial and commercial
underpinning of the modern business environment. This will be enabled through the
creation of an entrepreneur development cell on the campus.

 To encourage collaboration between KMEC and various organizations of repute on

various latest technological and research initiatives.

 To create a socially conscious and emotionally mature individuals who are aware of
India’s challenges and opportunities and who understand and appreciate the role and
responsibility of engineers towards achieving the goal of job and wealth creation.
 PROGRAM OUTCOMES(POs)

PO1. Engineering Knowledge: Apply the knowledge of mathematics, science,

Engineering fundamentals, and an engineering specialization to the solution of complex
engineering problems.

PO2. Problem Analysis: Identify formulate, review research literature, and analyze
complex engineering problems reaching substantiated conclusions using first principles of
mathematics, natural sciences, and engineering sciences

PO3. Design/Development of solutions: Design solutions for complex engineering

problems and design system components or processes that meet the specified needs with
appropriate consideration for the public health and safety, and the cultural, societal, and
environmental considerations.

PO4. Conduct Investigations of Complex problems: Use research-based knowledge and

research methods including design of experiments, analysis and interpretation of data, and
synthesis of the information to provide valid conclusions.

PO5. Modern Tool Usage: Create select, and, apply appropriate techniques, resources,
and modern engineering and IT tools including prediction and modeling to complex
engineering activities with an understanding of the limitations.

PO6. The Engineer and Society: Apply reasoning informed by contextual knowledge to
societal, health, safety. Legal und cultural issues and the consequent responsibilities
relevant to professional engineering practice.

PO7. Environment and Sustainability: Understand the impact of the professional

engineering solutions in societal and environmental contexts and demonstrate the
knowledge of, and need for sustainable development.

PO8. Ethics: Apply ethical principles and commit professional ethics and responsibilities
and norms of the engineering practice.
PO9. Individual and Team Work: Function effectively as an individual, and as a
member or leader in diverse teams and in multidisciplinary settings.

PO10. Communication: Communicate effectively on complex engineering activities with

the engineering community and with society at large, such as, being able to comprehend and
write effective reports and design documentation, make effective presentations, and give
and receive clear instructions.

PO11. Project Management and Finance: Demonstrate knowledge and understanding of

the engineering and management principles and apply these to one's own work, as a member
and leader in a team, to manage projects and in multidisciplinary environments.

PO12. Life-Long Learning: Recognize the need for, and have the preparation and ability
to engage in independent and life-long learning in the broadest context of technological
change.
 PROGRAM SPECIFIC OUTCOMES(PSOs)

PSO1. An ability to analyze the common business functions to design and develop
appropriate Information Technology solutions for social upliftment.

PSO2. Shall have expertise on the evolving technologies like Mobile Apps, CRM, ERP,
Big Data, etc.

PROGRAM EDUCATIONAL OBJECTIVES(PEOs)

PEO1. Graduates will have successful careers in computer related engineering fields or
will be able to successfully pursue advanced higher education degrees.

PEO2. Graduates will try and provide solutions to challenging problems in their
profession by applying computer engineering principles.

PEO3. Graduates will engage in life-long learning and professional development by

rapidly adapting to changing work environment.

PEO4. Graduates will communicate effectively, work collaboratively and exhibit high
levels of professionalism and ethical responsibility.
 PROJECT OUTCOMES

P1: Demonstrate the ability to synthesize and apply the knowledge and skills acquired in the
academic program to the real-world problems.
P2: Evaluate different solutions based on economic and technical feasibility
P3: Effectively plan a project and confidently perform all aspects of project management.
P4: Demonstrate effective coding, written, presentation and oral communication skills.

MAPPING PROJECT OUTCOMES WITH PROGRAM OUTCOMES

PO PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12

P1 H H M M M L L

P2 H M M L H L L

P3 M M H L M M M

P4 M L M L L L L

L– LOW M –MEDIUM H– HIGH

 PROJECT OUTCOMES MAPPING WITH PROGRAM SPECIFIC
OUTCOMES

PSO PSO1 PSO2

P1 H H

P2 H M

P3 M M

P4 M H

PROJECT OUTCOMES MAPPING WITH PROGRAM EDUCATIONAL

OBJECTIVES

PEO PEO1 PEO2 PEO3 PEO4

P1 H H M M

P2 M H L M

P3 M M H M

P4 M M M H
 DECLARATION

We hereby declare that the results embodied in the dissertation entitled “NEXT

WORD PREDICTOR'' has been carried out by us together during the academic year 2024-25

as a partial fulfillment of the award of the B.E degree in Computer Science & Engineering

from Osmania University. We have not submitted this report to any other university or

organization for the award of any other degree.

S. DHANUSH 245522733053

K. RAMCHARAN 245522733030

M. PRAVEEN 245522733035
 ACKNOWLEDGEMENT

We take this opportunity to thank all the people who have rendered their full
support to our project work. We render our thanks to Prof. P.V.N. Prasad, Principal,
who encouraged us to do the Project.

We are grateful to Mr. Neil Gogte, Founder & Director, and Mr. S. Nitin,
Director, for facilitating all the amenities required for carrying out this project.

We express our sincere gratitude to Ms. Deepa Ganu, Academic Director, for
providing an excellent environment in the college.

We are also thankful to Dr. CH. Rathan Kumar, Dr. Bala Krishna, Heads of
the Department for providing us with time to make this project a success within the
given schedule.

We are also thankful to Mrs.D.Mounika, Project Coordinator, for supporting us

from project selection till the submission to complete this project within the scheduled
time.

We are also thankful to our Faculty Supervisor Dr. / Mr. / Mrs. / Ms. XXXX,
for her/his valuable guidance and encouragement given to us throughout the project
work.

We would like to thank the entire CSE Department faculty, who helped us
directly or indirectly in the completion of the project.

We sincerely thank our friends and family for their constant motivation during
the project work.

S. DHANUSH 245522733053

K. RAMCHARAN 245522733030

M. PRAVEEN 245522733035
 ABSTRACT

In the era of intelligent systems and human-computer interaction, natural language

processing (NLP) has emerged as a vital domain enabling machines to understand and
generate human language. One of the most powerful advancements in this field is the
development of large language models like GPT-2, which have demonstrated significant
success in generating coherent and context-aware text.This project, titled next word
predictor using GPT-2, aims to build a smart word prediction tool that suggests the most
appropriate next word based on the partial sentence provided by the user. Unlike traditional
predictive models that focus on syntactic completion, this system emphasizes semantic
relevance by generating multiple possible continuations and filtering them using part-of-
speech (POS) tagging. Only meaningful word types such as nouns, verbs, adjectives, and
adverbs are considered for output.The implementation leverages the pre-trained GPT-2
model from Hugging Face’s Transformers library and uses the natural language toolkit
(NLTK) for linguistic processing, including stop word removal and POS tagging. The
system dynamically identifies the top three most contextually appropriate words from the
generated predictions. It supports execution on both CPU and GPU platforms, ensuring
accessibility for a wide range of users and developers.This project not only demonstrates
the real-world applicability of transformer-based models in enhancing typing interfaces
and writing assistants but also introduces a novel layer of linguistic refinement. By
focusing on meaningful predictions rather than raw generation, the system bridges the gap
between powerful language models and user-centric applications. The outcomes of this
work have the potential to improve the efficiency, coherence, and productivity of human-
machine communication systems.

i
 LIST OF FIGURES

Fig. No. Name of the Figure Page No.

ARCHITECTURE
1. 2
DIAGRAM

USE CASE
2. 7
DIAGRAM

ACTIVITY
3. 8
DIAGRAM

SEQUENCE
4. 9
DIAGRAM

5. CLASS DIAGRAM 10

ii
 CONTENTS
Abstract i
List of Figures ii
List of Tables iii

CHAPTER-1 INTRODUCTION 1
1.1 Purpose of the Project 1
1.2 Scope of the Project 1
1.3 Problem with existing systems 1
1.4 Architecture Diagram 1

CHAPTER-2 LITERATURE SURVERY 3

2.1 Overview 3
2.2 Comparative study of existing systems 4
2.2.1 GPT-2 Text Generator CLI 4
2.2.2 HumanEval + GPT-2 Benchmark 4

CHAPTER-3 SOFTWARE REQUIREMENT SPECIFICATION 5

3.1 Requirements 6
3.2 Functional Requirements 6
3.3 Non-Functional Requirements 6
3.4 Software Requirements 6
3.5 Hardware Requirements 7

CHAPTER-4 SYSTEM DESIGN 7

4.1 Introduction to UML 7

4.2 UML Diagrams 7
4.2.1 Use Case Diagram 7
4.2.2 Sequence Diagram 8
4.2.3 State Chart Diagram 9

4.2.4 Deployment Diagram 10

CHAPTER-5 IMPLEMENTATION
iii
 5.1 Code 11

CHAPTER-6 RESULTS 12

CHAPTER-7 CONCLUSION

7.1 Conclusion 14

REFERENCES

iv
 NEXT WORD PREDICTOR

CHAPTER-I
INTRODUCTION

1.1 Background

In today’s digital landscape, the interaction between humans and machines is becoming
increasingly natural and intelligent, driven by advancements in artificial intelligence (AI).
One of the most prominent and rapidly evolving fields within AI is natural language
processing (NLP), which focuses on enabling computers to understand, interpret, and generate
human language in a meaningful way.The emergence of transformer-based models has
significantly improved the capabilities of NLP applications. Among them, the generative pre-
trained transformer 2 (GPT-2), developed by OpenAI, has set a benchmark for generating
coherent and contextually appropriate text. GPT-2 is trained on a massive amount of internet
data and is capable of performing various language tasks with remarkable fluency.One
practical application of these language models is next word prediction, which plays a crucial
role in applications such as predictive text keyboards, writing assistants, and conversational
agents. However, traditional next word prediction systems often generate outputs based solely
on statistical likelihoods, which may not always align with semantic context or user intentTo
address this, the present project proposes a system that combines the generative strength of
GPT-2 with linguistic filtering techniques. The aim is to predict words that are not only
syntactically correct but also semantically meaningful—limited to parts of speech such as
nouns, verbs, adjectives, and adverbs.

1.2 Problem statement

Conventional next word predictors typically focus on the probability of word sequences based
on large corpora. While they may produce grammatically correct outputs, they often suggest
words that lack contextual or semantic value, such as conjunctions, prepositions, or filler
words.This project addresses the need for a more context-aware word prediction system that
provides outputs with higher semantic importance. By incorporating part-of-speech (POS)
tagging, the system filters out low-value words and retains only those that add meaning to the
sentence, thereby improving the quality and usefulness of the predictions.

1.3 Objectives
The primary objectives of this project are as follows:
 to implement a word prediction system using the pre-trained GPT-2 model
 to filter the generated outputs using POS tagging, thereby allowing only meaningful word types
such as nouns, verbs, adjectives, and adverbs
 to enhance the relevance of predicted words in the context of user input
 to improve typing efficiency and usability in applications that require next word suggestions

1.4 Scope of the project

The scope of this project is limited to predicting the next meaningful word based on a partial
sentence provided by the user. The project does not involve the training of new models but
makes use of an existing, pre-trained GPT-2 model.
Key functionalities covered in the scope include:loading and using GPT-2 for text generation
 generating multiple possible next words using sampling strategies such as top-k and top-p
 filtering generated words using NLTK-based POS tagging
 displaying the top three most contextually relevant and semantically meaningful words.

KESHAV MEMORIAL ENGINEERING COLLEGE 1

 NEXT WORD PREDICTOR

ARCHITECTURE DIAGRAM

KESHAV MEMORIAL ENGINEERING COLLEGE 2

 NEXT WORD PREDICTOR

CHAPTER-2
Literature Survey

2.1 Overview
In recent years, large language models have significantly transformed the field of natural
language processing. These models, including GPT-2, BERT, and T5, have demonstrated an
exceptional ability to generate, understand, and manipulate human language. Applications like
writing assistants, chatbots, and autocomplete systems are becoming increasingly
sophisticated due to these advancements.
This literature survey aims to analyze and compare previously developed word prediction and
language generation systems to identify techniques, evaluate their effectiveness, and highlight
the improvements implemented in the proposed project.

2.2 Comparative study of existing systems

2.2.1 GPT-2 by OpenAI (Author: OpenAI Team, Date: 14 Feb 2019)
Features:
 Transformer-based architecture
 Trained on large internet corpus
 Capable of generating fluent and context-aware text
 No fine-tuning needed for general tasks
What we took:
 Used GPT-2 as the core model for next word prediction
 Adopted pre-trained weights from Hugging Face
What we did differently:
 Added linguistic filtering for meaningful word suggestions
 Combined model predictions with POS tagging logic

2.2.2 GPT-2 Text Generator CLI (Author: Hugging Face Contributors, Date: 20 Nov
2020)
Features:
 Command-line text generation tool
 Based on Transformers library
 Simple prompt input and response output
What we took:
 Reference for prompt-token-output logic
 Usage of top-k and top-p sampling techniques
What we did differently:
 Added stop word filtering using NLTK
 Filtered only meaningful POS tags for output

2.2.3 Smart Compose (Google Research Paper, Date: 25 May 2018)

Features:
 Text completion in Gmail
 Predictive suggestions during typing
 Based on language modeling and ranking algorithms
What we took:
 Concept of generating multiple predictions
 Ranked output mechanism
What we did differently:
KESHAV MEMORIAL ENGINEERING COLLEGE 3
 NEXT WORD PREDICTOR

 Limited output to most meaningful 3 suggestions

 Emphasis on offline-friendly, lightweight setup

2.2.4 BERT-based Word Prediction (Author: Stanford NLP Group, Date: 18 Sep 2020)
Features:

2.2.5 AI21 Studio – Wordtune (Author: AI21 Labs, Date: 22 Dec 2021)
Features:
 Advanced rewrite and suggestion engine
 Provides alternative phrasing and word suggestions
 Real-time web-based system
What we took:
 Understanding of improving productivity through smart text suggestions
What we did differently:
 Built lightweight CLI tool instead of browser-based assistant
 Restricted output to single word suggestions only

2.2.6 HumanEval + GPT-2 Benchmark (Author: OpenAI GitHub Contributors, Date:

28 July 2021)
Features:
 Uses GPT-2 for evaluating human-like completions
 Benchmarks output fluency and contextual accuracy
What we took:
 Technique for evaluating generated content
 Use of multiple sequences to improve choice selection
What we did differently:
 POS-based semantic filtering of sequences
 Focus on meaningful language elements, not full sentence completion

KESHAV MEMORIAL ENGINEERING COLLEGE 4

 NEXT WORD PREDICTOR

CHAPTER 3
System Analysis

3.1 Existing system

Existing next word prediction systems are commonly integrated into mobile keyboards,
writing assistants, and chatbots. Most of these tools rely on statistical methods such as n-gram
models, or basic neural networks that predict the next probable word based on previous
inputs. While they can provide grammatically correct suggestions, their predictions often lack
semantic richness or relevance.
Many existing systems do not consider the contextual importance or grammatical category of
the predicted word. As a result, they may suggest common filler words such as articles,
conjunctions, or prepositions that do not add value to the user’s sentence. Additionally, these
systems often lack flexibility and are usually embedded within proprietary applications,
making customization and extension difficult.

3.2 Limitations of the existing system

 lack of semantic filtering: most systems do not filter predictions by parts of speech,
resulting in less meaningful suggestions
 low relevance: suggestions may be grammatically correct but not contextually
appropriate
 limited customization: users cannot easily modify the prediction logic or apply
linguistic constraints
 internet dependence: many systems rely on server-side processing, reducing offline
usability
 lack of transparency: users cannot observe or control the internal functioning of the
model

3.3 Proposed system

The proposed system leverages a pre-trained GPT-2 model for generating next word
predictions. Unlike conventional tools, it applies linguistic filters using natural language
toolkit (nltk) to retain only meaningful word types such as nouns, verbs, adjectives, and
adverbs. The system uses part-of-speech (POS) tagging and stop word removal to improve the
semantic value of the output.
The process begins with the user providing a partial sentence. The input is tokenized and
passed to GPT-2, which generates multiple possible continuations. These are then filtered
using nltk to remove irrelevant or unimportant words. Finally, the top three most relevant
suggestions are presented to the user.
the system supports execution on both cpu and gpu platforms and is implemented in python
using the transformers and nltk libraries.

KESHAV MEMORIAL ENGINEERING COLLEGE 5

 NEXT WORD PREDICTOR

3.4 Advantages of the proposed system

 context-aware predictions: output is filtered based on parts of speech to improve

relevance
 semantic value: suggestions include only meaningful word types, not just high-
frequency terms
 offline support: the system can run without an internet connection once dependencies
are downloaded
 modular design: easily extendable for future integration with graphical user interfaces
or web applications
 efficient and lightweight: supports execution on low-resource environments without
retraining the model

3.5 System requirements

hardware requirements:

 processor: intel core i3 or higher

 memory: minimum 4 gb ram
 storage: minimum 1 gb free disk space
 gpu (optional): nvidia cuda-enabled gpu for faster execution

software requirements:

 operating system: windows, linux or macos

 programming language: python 3.6 or above

libraries:

 transformers (hugging face)

 torch (pytorch)
 nltk
 collections (standard library)

KESHAV MEMORIAL ENGINEERING COLLEGE 6

 NEXT WORD PREDICTOR

CHAPTER 4
System Design

4.1 INTRODUCTION TO UML

Unified Modeling Language (UML) is a standardized modeling language used in software

engineering to visualize the structure and design of a system. It helps in understanding,
designing, and documenting software systems from different perspectives. In our project
“Personal Voice Assistant – Yaara”, UML diagrams help us visualize the system's
architecture, interactions, and behaviors before implementation.

4.2 UML DIAGRAMS

USECASE DIAGRAM

The Use Case Diagram captures the functionalities that the system offers from a user’s
perspective. It outlines the interaction between the user and the voice assistant.

KESHAV MEMORIAL ENGINEERING COLLEGE 7

 NEXT WORD PREDICTOR

Activity Diagram:

Shows the dynamic flow of activities or actions in a system or process. It’s useful for
modeling workflows, business processes, or use case behavior.

KESHAV MEMORIAL ENGINEERING COLLEGE 8

 NEXT WORD PREDICTOR

Sequence Diagram:

Illustrates how objects interact with each other in a particular sequence over time. It focuses
on the order of messages exchanged between objects.

Class Diagram:

Describes the structure of classes in a system including their fields and functions. Also
displays relationships like inheritance, association, and dependencies.

KESHAV MEMORIAL ENGINEERING COLLEGE 9

 NEXT WORD PREDICTOR

KESHAV MEMORIAL ENGINEERING COLLEGE 10

 NEXT WORD PREDICTOR

CHAPTER 5
IMPLEMENTATION
6.1 INTRODUCTION

This chapter presents the implementation details of the proposed system — a next word
predictor based on GPT-2 and linguistic filtering using NLTK. The implementation is carried
out using Python, with the integration of the Hugging Face Transformers library for GPT-2
and the Natural Language Toolkit (NLTK) for linguistic processing. The system is structured
to be simple, modular, and efficient.

6.2 CODE

from collections import Counter

import nltk
import torch
from nltk import pos_tag, word_tokenize
from nltk.corpus import stopwords
from transformers import GPT2LMHeadModel, GPT2Tokenizer

# One-time download
nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')
nltk.download('stopwords')

# Load GPT-2
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
model.eval()

# Device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

# Stopwords to ignore
stop_words = set(stopwords.words("english"))

# Only consider these POS tags (verbs, nouns, adjectives, adverbs)

allowed_tags = {"VB", "VBD", "VBG", "VBN", "VBP", "VBZ", "NN", "NNS", "JJ", "RB"}

print("✅ GPT-2 ready. Meaningful predictor running!")

def get_next_meaningful_words(prompt):
inputs = tokenizer(prompt, return_tensors="pt").to(device)

outputs = model.generate(
**inputs,
max_new_tokens=5,
num_return_sequences=20,
do_sample=True,
KESHAV MEMORIAL ENGINEERING COLLEGE 11
 NEXT WORD PREDICTOR

temperature=1.0,
top_k=50,
top_p=0.95,
pad_token_id=tokenizer.eos_token_id
)

next_words = []

for output in outputs:

decoded = tokenizer.decode(output, skip_special_tokens=True)
continuation = decoded[len(prompt):].strip()
first_word = continuation.split()[0] if continuation else ""

# Clean and POS tagging

if first_word.isalpha() and first_word.lower() not in stop_words:
tagged = pos_tag([first_word])
tag = tagged[0][1]
if tag in allowed_tags:
next_words.append(first_word.lower())

top3 = [word for word, _ in Counter(next_words).most_common(3)]

return top3

# Input loop
while True:
prompt = input("\n🗣 Enter a sentence (or type 'exit' to quit):\n>> ").strip()

if prompt.lower() == "exit":
print("\n🙏 Exiting... Har Har Mahadev!")
break

top3 = get_next_meaningful_words(prompt)
print("\n🔥 Top 3 Meaningful Next Words:")
print(top3 if top3 else ["No strong prediction. Try a different sentence."])

6.3 TECHNOLOGY STACK

 language: python 3.10

 ide: visual studio code
 nlp libraries: nltk, transformers
 deep learning framework: pytorch
 model used: gpt-2 (via hugging face)
 supporting libs: collections, torch
 hardware: supports cpu and gpu
 functions used: pos tagging, stopword removal, text generation

KESHAV MEMORIAL ENGINEERING COLLEGE 12

 NEXT WORD PREDICTOR

CHAPTER 6
RESULTS

6.1 SCREENSHOTS

6.1.1 Succesful Nltk Downloads

6.1.2 Model Load Confirmation

6.1.3 User Input

6.1.4 Predicted Word

6.1.5 Error Handling

6.1.6 Exit

KESHAV MEMORIAL ENGINEERING COLLEGE 13

 NEXT WORD PREDICTOR

CHAPTER 7
CONCLUSION

7.1 CONCLUSION

The project titled "Next Word Predictor using GPT-2" successfully demonstrates how
state-of-the-art transformer-based language models can be integrated with linguistic filters to
provide semantically meaningful word predictions. Unlike conventional predictive text
systems that primarily rely on statistical or shallow neural techniques, this system leverages
the deep contextual understanding of GPT-2 along with NLTK-based linguistic constraints
to ensure the output suggestions are not only grammatically valid but also contextually
relevant.
The architecture was designed to be flexible, lightweight, and efficient. It supports execution
on both CPU and GPU platforms, making it accessible to a wide range of users and
developers. By filtering GPT-2’s generated outputs using part-of-speech (POS) tagging and
excluding irrelevant or filler words through stop word removal, the system enhances both the
quality and usefulness of the predictions. This contributes significantly to applications such as
typing assistants, writing enhancers, and conversational AI interfaces.
The core contribution of this project lies in its hybrid approach, which combines deep
learning with rule-based linguistic filtering. This results in more human-like interactions,
improving not just prediction accuracy but also the end-user experience. The successful
integration of the Hugging Face Transformers library and NLTK demonstrates how modern
open-source tools can be effectively used to build intelligent, personalized applications.
Furthermore, the system’s modularity makes it a strong foundation for future enhancements,
such as GUI integration, speech-to-text pipelines, multi-language support, and offline
capabilities. Overall, this project stands as a practical example of applying advanced Natural
Language Processing (NLP) in real-world utility, proving that the synergy between AI and
linguistics can significantly enhance human-computer communication.

KESHAV MEMORIAL ENGINEERING COLLEGE 14

 NEXT WORD PREDICTOR

REFERENCES

[1] Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I.; Improving Language
Understanding by Generative Pre-Training; OpenAI, June 2018;
https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_pape
r.pdf

[2] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... &
Polosukhin, I.; Attention is All You Need; Proceedings of the 31st International Conference on
Neural Information Processing Systems (NIPS), 2017, pp. 5998–6008;
https://doi.org/10.48550/arXiv.1706.03762

[3] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., ... & Rush, A. M.;
Transformers: State-of-the-Art Natural Language Processing; Proceedings of the 2020
Conference on Empirical Methods in Natural Language Processing: System Demonstrations
(EMNLP 2020), October 2020, pp. 38–45; https://doi.org/10.18653/v1/2020.emnlp-demos.6

[4] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K.; BERT: Pre-training of Deep
Bidirectional Transformers for Language Understanding; Proceedings of the 2019
Conference of the North American Chapter of the Association for Computational Linguistics
(NAACL), June 2019, pp. 4171–4186; https://doi.org/10.48550/arXiv.1810.04805

[5] Hugging Face Documentation; Transformers Library - GPT-2; HuggingFace.co, Accessed

July 2025; https://huggingface.co/transformers/model_doc/gpt2.html

[6] Loper, E., & Bird, S.; NLTK: The Natural Language Toolkit; Proceedings of the ACL-02
Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing
and Computational Linguistics, July 2002, pp. 63–70;
https://doi.org/10.3115/1118108.1118117

KESHAV MEMORIAL ENGINEERING COLLEGE 15