Manideep Lenkalapally

SUMMARY:
• Over 12+ years of overall software development experience on Big Data Technologies, Hadoop
Ecosystem and Java/J2EE Technologies.
• Worked on Data Modeling using various ML (MachineLearningAlgorithms) via R and
Python (Graphlab) Worked on Programming Languages like CoreJava and Scala.
• In depth and extensive knowledge of Hadoop Architecture and various components such as
HDFS, Job Tracker, Task Tracker, Name Node, Data Node, Yarn, Resource Manager,
Node Manager and Map Reduce.
• Experience in Amazon AWS cloud which includes services like: EC2, S3, EBS, ELB, AMI,
Route53, Autoscaling, CloudFront, CloudWatch, Security Groups.
• A very good understanding of job workflow scheduling and monitoring tools like Oozie and
Control-M.
• Worked on developing ETL processes to load data from multiple data sources to HDFS using
Flume and Sqoop, perform structural modifications using MapReduce, Hive and analyze
data using visualization/reporting tools.
• Designed, configured and deployed Amazon Web Services (AWS) for a multitude of
applications utilizing the AWS stack (IncludingEC2, Route53, S3, RDS, Cloud Formation,
Cloud Watch, SQS, IAM), focusing on high-availability, fault tolerance, and auto-scaling.
• Worked on HDFS,Name Node, Job Tracker, Data Node, Task Tracker and the
MapReduce concepts.
• Experience in Front-end Technologies like Html, CSS, Html5, CSS3, and Ajax.
• Experience in building high performance and scalable solutions using various Hadoop
ecosystem tools like Pig, Hive, Sqoop, Spark, Zookeeper, Solr and Kafka.
• Defined real time data streaming solutions across the cluster using Spark Streaming,
Apache Storm, Kafka, Nifi and Flume.
• Solid Experience in optimizing the Hive queries using Partitioning and
Bucketingtechniques, which controls the data distribution, to enhance performance.
• Experience in Importing and Exportingdata from different databases like MySql, Oracle into
Hdfs and Hive using Sqoop.
• Expertise with Application servers and web servers like Oracle WebLogic, IBM WebSphere
and Apache Tomcat.
• Experience working in environments using Agile (scrum) and Waterfall methodologies.
• Expertise in database modeling and development using Sqland PL/SQL, MySql, Teradata.
• Experience in Amazon web services (AWS) and Google Cloud Platform (GCP).
• Experienced in Data Bricks, Hive SQL, Azure CI/CD pipeline, Delta Lake, Hadoop file system,
Snowflake.
• Worked on Apache Filnk to implement the tranformation on Data streams for
filtering,aggregating, update state.
• Experienced on Spark Architecture including Spark Core, Spark SQL, Data Frames,
Spark Streaming and Spark MLlib.
• Experience in NoSQLDatabasesHBase, Cassandra and it's integrated with Hadoop cluster.
• Experienced on cloud integration with AWS using Elastic Map Reduce (EMR), Simple
Storage Service (S3), EC2, Redshift and Microsoft Azure.
• Experienced on different Relational Data Base Management Systems like Teradata,
PostgreSQL, DB2, Oracle and SQL Server.
• Experience in building, deploying and integrating applications in Application Servers with
ANT, Maven and Gradle.
• Significant application development experience with REST Web Services, SOAP, WSDL,
and XML.
• Hands on experience in Azure Cloud, Azure DevOps, Azure Data Factory, Azure Data Lake
Storage, Azure Cosmos NO SQL DB, Azure HD Insight Big Data Technologies (Hadoop and
Apache Spark) and Data bricks
• Skilled in working with Hive data warehouse and snowflake modelling  
• Expertise in Database Design, Creation and Management of Schemas, writing Stored
Procedures, Functions, DDL and DML SQL queries and writing complex queries for Oracle
 • Strong hands-on development experience with Java, J2EE (Servlets, JSP, Java Beans,
EJB, JDBC, JMS, Web Services) and related technologies.
• Experience in working with different data sources like Flat files, XML files and Databases.
• Experience in database design, entity relationships, database analysis, programming SQL,
storedprocedure'sPL/ SQL, packages and triggers in Oracle and MongoDB on
Unix/Linux.
• Worked on different operating systems like UNIX/Linux, Windows XP and Windows 7,8,10.

TECHNICAL SKILLS
• Big data/Hadoop: Hadoop 3.0, HDFS, MapReduce, Hive 2.3, Pig 0.17, Sqoop 1.4, Oozie
4.3, Hue, Flume 1.8, Kafka 1.1 and Spark,Apache Flink
• NoSQL Databases: HBase, Cassandra, MongoDB 3.6
• Cloud Technology: Amazon Web Services (AWS), EC2, EC3, Elastic Search, Microsoft
Azure.
• Languages: Java, J2EE, PL/SQL, Pig Latin, HQL, R 3.5, Python 3.6, XPath
• Java Tools & Web Technologies: EJB, JSF, Servlets, JSP, JSTL, CSS3/2, HTML5.5,
XHTML, CSS, XML, XSL, XSLT
• Databases: Oracle12c/11g, MYSQL, DB2, MS SQL Server 2016/2014
• Frame Works: Struts, Spring 5.0, Hibernate 5.2, MVC
• Web Services: SOAP, Restful, JAX-WS, Apache Axis
• Application Server: Apache Tomcat 9.0.8, Jboss, IBM Web sphere, Web Logic
• Scripting Languages: Shell Scripting, Java Script.
• Tools and IDE: SVN, Maven, Gradle, Eclipse 4.7, Netbeans 8.2
• Open Source: Hibernate, Spring IOC, Spring MVC, Spring Web Flow, Spring AOP
• Methodologies: Agile, RAD, JAD, RUP, Waterfall & Scrum

WORK EXPERIENCE

Disney Orlando, FL
Aug 2020 to Present
Sr. Big Data Engineer
Responsibilities:
• Responsible for building scalable distributed data solutions using Hadoop Cloudera.
• Involved in story-driven agile development methodology and actively participated in daily scrum
meetings.
• Created RDD's and applied data filters in Spark and created Cassandra tables and Hive tables for user
access.
• Involved in development of real time streaming applications using Pyspark, Apache Flink, Kafka, Hive
on distributed Hadoop Cluster.
• Created Partitioning, Bucketing, and Map Side Join, Parallel execution for optimizing the hive queries
decreased the time of execution from hours to minutes.
• Designed AWS, Cloud migration, AWS EMR, Dynamo DB, Redshift and event processing using
lambda function.
• Worked with Amazon EMR to process data directly in S3 when we want to copy data from S3 to the
Hadoop Distributed File System (HDFS)on your Amazon EMR cluster by setting up the Spark Core for
analysis work.
• Worked on importing data from MySQLDB to HDFSand vice-versa using Sqoop to configure Hive
Metastore with MySQL, which stores the metadata for Hive tables.
• Worked with NoSQL databases like HBase in creating HBase tables to load large sets of semi-
structured data coming from various sources.
• Worked with different actions in Oozie to design workflow like Sqoopaction, pig action, hive action,
shell action.
• Install and configure Apache Airflow for S3 bucket and Snowflake data warehouse and created dags to
run the Airflow.
 • Loaded salesforce Data every 15 min on incremental basis to BIGQUERY raw and UDM layer using
SOQL, Google DataProc, GCS bucket, HIVE, Spark, Scala, Python, Gsutil and Shell Script.
• Experienced in Developing Spark applications using Spark - SQL, Pyspark and Delta Lake in Databricks
for data extraction, transformation, and aggregation from multiple file formats for analyzing &
transforming the data to uncover insights into the customer usage patterns.
• Mastered major Hadoop distributes like Hortonworks and Cloudera numerous Open Source projects
and prototype various applications that utilize modern Big Data tools.
• Analyzed large and critical datasets using Cloudera, HDFS, HBase, MapReduce, Hive UDF, Pig,
Sqoop, Zookeeper and Spark.
• Built a Scala and spark based configurable framework to connect common Data sources like MYSQL,
Oracle, Postgres, SQL Server, Salesforce, Bigquery and load it in Bigquery.
• Developed Hive Scripts, Pig scripts, Unix Shell scripts, Spark programming for all ETL loading
processes and converting the files into parquet in the Hadoop File System.
• Created applications using Kafka, which monitors consumer lag within Apache Kafka clusters.
• Involved in converting Hive/SQL queries into Spark transformations using SparkData Frames and
Scala.
• Used AWS Glue for the data transformation, validate and data cleansing.
• Developed PySpark code to mimic the transformations performed in the on-premise environment.  
• Loaded and transformed large sets of structured, semi structured data through Sqoop.
• Optimizing existing algorithms in Hadoop using Spark Context, Hive-SQL, and Data Frames.
• Worked on custom Pig Loaders and Storage classes to work with a variety of data formats such as
JSON, Compressed CSV, etc.
• Used Flink Streaming for pipelined Flink engine to process data streams to deploy new API including
definition of flexible windows.
• Used AWS glue catalog with crawler to get the data from S3 and perform sql query operations
• Developed and deployed the outcome using spark and Scala code in Hadoop cluster running on GCP.
• Created various hive external tables, staging tables and joined the tables as per the requirement.
Implemented static Partitioning, Dynamic partitioning and Bucketing. Install and configure Apache
Airflow for S3 bucket and Snowflake data warehouse and created dags to run the Airflow.
• Developed Shell, Perl and Pythonscripts to automate and provide Control flow to Pig scripts. Design of
RedshiftData model, Redshift Performance improvements/analysis
• Implemented solutions for ingesting data from various sources and processing the Data-at-Rest
utilizing Big Data technologies such as Hadoop, Map Reduce Frameworks, HBase, Hive.
• Worked using Apache Hadoop ecosystem components like HDFS, Hive, Sqoop, Pig, and Map Reduce,
Worked with Spark and Python.
• Experienced in Data Bricks, Hive SQL, Azure CI/CD pipeline, Delta Lake, Hadoop file system,
Snowflake.
• Developed Spark applications using Spark - SQL in Databricks for data extraction.
• Launched multi-node Kubernetes cluster in Google Kubernetes Engine (GKE) and migrated the
dockerized application from AWS to GCP.
• Created HBase tables to load large sets of structured, semi-structured and unstructured data coming
from UNIX, NoSQL and a variety of portfolios
Environment: Hadoop 3.0, Cassandra 3.11, Hive 2.3, Redshift,HDFS, MySQL 8.2, Sqoop 1.4,NoSQL, Oozie
4.3, pig 0.17, Hortonworks, Apache Flink, Snowflake, MapReduce, HBase, Zookeeper 3.4, AWS Glue, Spark,
Unix, Kafka, JSON, Python 3.6

American Express, phoenix AZ

Mar 18 – Aug 20
Big Data Engineer
Responsibilities:
• Responsible for building scalable distributed data solutions using spark and Hadoop. Used Solid
Understanding of Hadoop HDFS, Map-Reduce and other Ecosystem Projects.
• Worked on analyzing Hadoopcluster using different big data analytic tools including Kafka, Pig, Hive
and MapReduce.
• Designed, developed Hadoop eco system components, installation, configuration, supporting and
monitoring of Hadoop clusters using Apache, Cloudera distributions, Azure data bricks and AWS.
• Involved in story-driven agile development methodology and actively participated in daily
scrummeetings.
 • Working on both kind of data processing as batch and streaming with ingestion to NoSQL and
HDFSwith different file format such as parquet and AVRO.
• Developed multiple Kafka Producers and Consumers as per the business requirement also customized
the partition to get optimized results.
• Involved on configuration, development of Hadoop environment with AWS cloud such as EC2, EMR,
Redshift, Cloud watch, and Route.
• Developed a detailed project plan and helped manage the data conversion migration from the legacy
system to the target snowflake database.
• Developed the batch scripts to fetch the data from AWS S3 storage and do required transformations in
Scala using sparkframework.
• Configured Spark streaming to receive real time data from the Kafka and store the stream data to
HDFSusing Scala.
• Developed common Flink module for serializing and deserializing AVRO data by applying schema.
• Developed pipeline for POC to compare performance/efficiency while running pipeline using the AWS
EMR Spark cluster and Cloud Dataflow on GCP.
• Responsible for developing data pipeline using flume, Sqoop and Pig to extract the data from weblogs
and store in HDFS.
• Responsible for creating on-demand tables on S3 files using Lambda Functions and AWS Glue using
Python and Pyspark.
• Expertise in building and architecting multiple Data pipelines, end to end ETL and ELT process for
Data ingestion and transformation in GCP and coordinate task among the team.
• Worked on migrating MapReduce programs into Spark transformations using Spark and Scala, initially
done using python (Pyspark).
• Developed Pig Latin scripts to extract data from the web server output files to load into HDFS.
• Developed data pipeline using MapReduce, Flume, Sqoop and Pig to ingest customer behavioral data
into HDFSfor analysis.
• Used Azure Portal, Azure PowerShell, Storage Accounts, and Azure Data Management.
• Used Spark for interactive queries, processing of streaming data and integration with popular
NoSQLdatabase for huge volume of data.
• Used the Spark -Cassandra Connector to load data to and from Cassandra.
• Handled importing data from different data sources into HDFSusing Sqoop and also performing
transformations using Hive, MapReduce and then loading data into HDFS.
• Exported the analyzed data to the relational databases using Sqoop, to further visualize and generate
reports for the BI team.
• Collecting and aggregating large amounts of log data using Flume and staging data in HDFSfor further
analysis
•
• Analyzed the data by performing Hive queries (HiveQL) and running Pigscripts (PigLatin) to study
customer behavior.
• Deployed application to GCP using Spinnaker (rpm based)
• Extracted large volumes of data feed on different data sources, performed transformations and loaded
the data into various Targets.
• Developed data formatted web applications and deploy the script using HTML5, XHTML, CSS, and
Client- side scripting using JavaScript.
• Involved in loading and transforming large sets of Structured, Semi-Structured and Unstructured data
and analyzed them by running Hive queries and Pig scripts
• Assisted in Cluster maintenance, Cluster Monitoring, and Troubleshooting, Manage and review data
backups and log files.
Environment: Hadoop 3.0, Pig 0.17, Hive 2.3, HBase, Oozie 4.1, Sqoop 1.4, Kafka 1.1, Spark, Impala,
HDFS, MapReduce, Redshift, Scala, flume, NoSQL, Cassandra 3.11, XHTML, AWS Glue, CSS3,
HTML5, JavaScript

Client: ONSOLVE, Daytona beach, FL

Sep 16 – Feb 18
Hadoop Developer
Responsibilities:
• Involved in Installing HadoopEcosystem components.
• Involved in HDFS maintenance and administering it through Hadoop - Java API.
 • Analyzed the data using Spark, Hive and produced summary results to downstream systems.
• Created Shell scripts for scheduling data cleansingscripts and ETL loading process.
• Installed and Configured multi-nodes fully distributed Hadoop cluster.
• Analyzed large and critical datasets using Cloudera, HDFS, HBase, MapReduce, Hive, UDF, Pig,
Sqoop, Zookeeper and Spark.
• Designed and implemented MapReduce based large-scale parallel relation-learning system.
• Developed and delivered quality services on-time and on-budget. Solutions developed by the team use
Java, XML, HTTP, SOAP, Hadoop, Pig and other webtechnologies.
• Created the JDBC data sources in the Weblogic.
• Used the existing database reference tables in order for consumption using JDBC mapping.
• Used Html, CSS, JDBCDriver, JSP, AJAX, GoogleAPI and Webmashup.
• Involved in end to end data processing like ingestion, processing, and quality checks and splitting.
• Imported data into HDFS from various SQLdatabases and files using Sqoop and from streaming
systems using Storm into Big Data Lake.
• Involved in scripting (python and shell) to provision and spin up virtualized Hadoopclusters.
• Worked with NoSQLdatabases like Base to create tables and store the data Collected and aggregated
large amounts of log data using ApacheFlume and staged data in HDFS for further analysis.
• Developed custom aggregate functions using SparkSQL and performed interactive querying.
• Wrote Pigscripts to store the data into HBase
• Created Hive tables, dynamic partitions, buckets for sampling, and worked on them using HiveQL
• Exported the analyzed data to Teradata using Sqoop for visualization and to generate reports for the BI
team. Experienced on loading and transforming of large sets of structured, semi structured and
unstructured data.
• Developed the code which will create XMLfiles and Flat files with the data retrieved from Databases
and XMLfiles.
• Extracted files from RDBMS through Sqoop and placed in HDFS and processed.
• Spark Streaming collects this data from Kafka in near-real-time and performs necessary
transformations and aggregation on the fly to build the common learner data model and persists the
data in NoSQL store (HBase).
• Configured Fair Scheduler to provide service level agreements for multiple users of a cluster.
• Loaded data into the cluster from dynamically generated files using FLUME and from RDBMS using
Sqoop.
• Involved in writing JavaAPI's for interacting with HBase
• Involved in writing Flume and Hivescripts to extract, transform and load data into Database
• Participated in development/implementation of ClouderaHadoop environment.
• Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for efficient data access.
• Load and transform large sets of structured, semi structured and unstructured data using Hadoop/Big
Data concepts.
• Ingested semi structured data using Flume and transformed it using Pig.
Environment: Cloudera, HDFS, HBase, MapReduce, Hive 2.0, UDF, Pig 0.16, Sqoop 1.1, Zookeeper 2.8,
Spark, RDBMS, Kafka 1.0, Teradata r13, Java, XML, HTTP, SOAP, Hadoop 2.8, and Flume

Client: Huntington National bank, Chicago, IL

Apr 2013 – Sep 2016
Role: Sr. Java/J2EE Developer
Responsibilities:
• Responsible for designing Rich user Interface Applications using JavaScript, CSS, HTML and Ajax
and developed web services by using SOAPUI.
• Applied J2EE Design Patterns such as Factory, Singleton, and Business delegate, DAO, Front
Controller Pattern and MVC.
• Created POJO layer to facilitate the sharing of data between the front end and the J2EE business
objects.
• Implemented Log4j by enabling logging at runtime without modifying the application binary.
• Provided ANT build script for building and deploying the application.
• Involved in configuring and deploying the application on WebLogicApplicationServer.
• Used CVS for maintaining the Source Code Designed, developed and deployed on Apache Tomcat
Server.
• Created and modified Stored Procedures, Functions, Triggers and Complex SQL Commands using
PL/SQL.
 • Developed Shell scripts in Unix and procedures using Sql and PL/Sql to process the data from the input
file and load into the database.
• Designed and develop web based applications using HTML5, CSS, JavaScript (JQuery), AJAX, and
JSP framework.
• Involved in the migration of build and deployment process from ANT to Maven.
• Developed Custom Tags to simplify the JSP code. Designed UI Screens using JSP, Struts tags and
HTML.
• Developed a multi-user web application using JSP, Servlets, JDBC, Spring and Hibernateframework to
provide the needed functionality.
• Used JSP, JavaScript, Bootstrap, JQuery, AJAX, CSS3, and HTML4 as data and presentation.
• Involved in J2EE Design Patterns such as Data Transfer Object (DTO), DAO, Value Object and
Template.
• Developed SQL Queries for performing CRUD operations in Oracle for the application.
• Implemented modules using Java APIs, Java collection, Threads, XML, and integrating the modules.
• Developed the presentation layer GUI using JavaScript, JSP, HTML, XHTML, CSS, custom tags and
developed Client-Side validations using Struts validate framework.
• Worked on UML diagrams like Class Diagram, Sequence Diagram required for implementing the
Quartz scheduler.
• Extensively used Eclipse IDE for developing, debugging, integrating and deploying the application.
• Managing and maintaining NoSQL database mainly MongoDB and used Multithreading at back-end
components in production domain.
• Extensively used Java Multi-Threading concept for downloading files from a URL.
Environment: CSS2, HTML4, Ajax, PL/SQL, UNIX, Sql, Hibernate3, Oracle10g, Maven, JavaScript,
Spring MVC

Client : Cisco, Raleigh nc

Jul 2011 – Apr 2013
Role: Java Developer
Responsibilities:
• Designed and developed java backend batch jobs to update the product offer details Core Java coding
and development using Multithreading and Design Patterns.
• Used Spring MVC framework to develop the application and its architecture
• Used spring dependency injection to inject all the required dependency in application.
• Developed screens, Controller classes, business services and Dao layer respective to the modules.
• Involved in developing the Business Logic using POJOs
• Developed Graphical User Interfaces using HTML and JSP's for user interaction
• Developed web pages using UIframeworksAngularJS.
• Created set of classes using DAO pattern to decouple the business logic and data
• Implemented Hibernate in the data access object layer to access and update information in the Sql
Server Database
• Used various Core Java concepts such as Multi-Threading, Exception Handling, Collection APIs to
implement various features and enhancements
• Wrote test cases in JUnit for unit testing of classes
• Interfaced with the Oracle back-end database using Hibernate Framework and XML configured files
• Created dynamic HTML pages, used JavaScript for client-side validations, and AJAX to create
interactive front-end GUI.
• Consumed Web Services for transferring data between different applications
• Used Restful Web services to retrieve credit history of the applicants
• Involved in coding, maintaining, and administering Servlets and JSP components to be deployed on a
spring boot.
• Wrote PL/SQL queries, stored procedures, and triggers to perform back-end database operations.
• Built scripts using Maven to build the J2EE application.
• Used Eclipse IDE for developing code modules in the development environment
• Performed connectivity with Sql database using JDBC.
• Implemented the logging mechanism using Log4jframework
• Used GITversion control to track and maintain the different version of the application.

Environment: Java 1.2, spring, HTML4, AngularJS, Hibernate, Oracle9i, AJAX, PL/SQL, Maven, J2EE,
EclipseIDE, Sql