sqoop jobs in greater noida

35 Sqoop Jobs in Greater Noida

Toggle to save search
posted 2 months ago
experience3 to 7 Yrs
location
Karnataka
skills
  • SQL
  • Python
  • Alteryx
  • Tableau
  • MicroStrategy
  • Cognos
  • Business Intelligence
  • Data Mining
  • Predictive Analytics
  • Data Science
  • Process Mining
  • Project Management
  • Hadoop ecosystem
  • Data Modelling
  • Solution Development Life Cycle
Job Description
As a Business Analytics Analyst (Officer) in the Internal Audit Analytics Team at Citi, your role involves working with the Digital Solutions and Innovation (DSI) team within the Citi Internal Audit Innovation function to identify opportunities, design, develop, and implement analytics to support audit activities. Your proficiency in analytics technology and tools will contribute to enhancing audit efficiency and effectiveness while having a functional knowledge of banking processes and related risks and controls. **Key Responsibilities:** - Participate in the innovative use of audit analytics through direct involvement in all audit phases. - Support defining data needs, designing, and executing audit analytics in alignment with audit methodology and professional standards. - Assist in executing automated routines to streamline audit testing. - Implement innovative solutions and predefined analytics following standard A&A procedures. - Support audit teams in conducting moderately complex audits in specific banking areas. - Collaborate with the Analytics and Automation team and wider Digital Solutions and Innovation team. - Utilize strong communication skills to articulate analytics requirements and results clearly. - Develop professional relationships with audit teams to identify analytics and automation opportunities. - Foster effective working relationships with technology and business teams for process understanding and data sourcing. - Promote continuous improvement in audit automation activities. **Key Qualifications and Competencies:** - Minimum 3 years of business/audit analyst experience applying analytical techniques and automated solutions. - Work experience in a global environment and large company setting. - Excellent technical, programming, and database skills. - Strong analytical ability to comprehend business processes, risks, controls, and develop innovative audit analytics. - Interpersonal and multicultural skills for engaging with internal and external audit stakeholders. - Self-driven with a problem-solving approach and adherence to procedures. - Detail-oriented with a focus on work product quality, data completeness, and accuracy. - Data literate with the ability to communicate effectively with technical and non-technical stakeholders. **Technical Skills Proficiency:** - SQL - Python - Hadoop ecosystem (Hive, Sqoop, PySpark, etc.) - Alteryx **Data Visualization Tools:** Proficiency in at least one of the following: - Tableau - MicroStrategy - Cognos **Experience in the Following Areas is a Plus:** - Business Intelligence, statistics, data modeling, data mining, and predictive analytics. - Application of data science tools and techniques. - Working with non-structured data like PDF files. - Banking Businesses or areas of expertise. - Big Data analysis including HUE, Hive. - Project Management/Solution Development Life Cycle. - Exposure to Process mining software such as Celonis. **What we Offer:** - Development in an innovative environment with the latest technologies. - Professional growth in a global setting. - Inclusive corporate culture promoting gender diversity and equality. - Supportive workplace for professionals returning from childcare leave. - Challenging learning path for a deep understanding of Citis products and services. - Yearly discretionary bonus and competitive social benefits. This job description provides an overview of the work performed, and other job-related duties may be assigned as needed.,
ACTIVELY HIRING

Top Companies are Hiring in Your City

For Multiple Roles

Jio Platforms Ltd
Jio Platforms Ltdslide-preview-Genpact
posted 1 month ago
experience5 to 12 Yrs
location
Maharashtra, Navi Mumbai
skills
  • Pentaho
  • Talend
  • PostgreSQL
  • Oracle
  • Vertica
  • Kafka
  • Airflow
  • Autosys
  • Git
  • Bitbucket
  • Tableau
  • Cognos
  • Python
  • Perl
  • GCP
  • Hadoop
  • Hive
  • Spark
  • Sqoop
  • Agile Environment
  • Dimensional Modeling
  • Documentation
  • MSSQL
  • Redshift
  • BigQuery
  • ControlM
  • ER Diagrams
  • Strong Communication
Job Description
As a Data Engineering Senior Software Engineer / Tech Lead / Senior Tech Lead at a leading digital health platform in Navi Mumbai, Ghansoli, you will play a crucial role in designing, building, and optimizing ETL/ELT pipelines using tools like Pentaho, Talend, or similar. Your responsibilities will include working on traditional databases such as PostgreSQL, MSSQL, Oracle, as well as MPP/modern systems like Vertica, Redshift, BigQuery, and MongoDB. You will collaborate cross-functionally with BI, Finance, Sales, and Marketing teams to define data needs and participate in data modeling, data quality checks, and data integration. Additionally, you will be implementing solutions involving messaging systems like Kafka, REST APIs, and scheduler tools such as Airflow, Autosys, or Control-M while ensuring code versioning and documentation standards are followed using Git or Bitbucket. Key Responsibilities: - Design, build, and optimize ETL/ELT pipelines using tools like Pentaho, Talend, or similar - Work on traditional databases (PostgreSQL, MSSQL, Oracle) and MPP/modern systems (Vertica, Redshift, BigQuery, MongoDB) - Collaborate cross-functionally with BI, Finance, Sales, and Marketing teams to define data needs - Participate in data modeling (ER/DW/Star schema), data quality checks, and data integration - Implement solutions involving messaging systems (Kafka), REST APIs, and scheduler tools (Airflow, Autosys, Control-M) - Ensure code versioning and documentation standards are followed (Git/Bitbucket) Qualifications Required: - ETL Tools: Pentaho / Talend / SSIS / Informatica - Databases: PostgreSQL, Oracle, MSSQL, Vertica / Redshift / BigQuery - Orchestration: Airflow / Autosys / Control-M / JAMS - Modeling: Dimensional Modeling, ER Diagrams - Scripting: Python or Perl (Preferred) - Agile Environment, Git-based Version Control - Strong Communication and Documentation In this role, you will have the opportunity to work on hands-on development of ETL pipelines, data models, and data inventory as a Senior Software Engineer. As a Tech Lead, you will lead mid-sized data projects and small teams, deciding on ETL strategy and performance tuning. As a Senior Tech Lead, you will drive data architecture decisions, oversee large-scale data ingestion, and mentor junior leads while owning stakeholder delivery end-to-end. Experience with AdTech/Marketing data and the Hadoop ecosystem would be advantageous for the Senior Tech Lead role. Join this dynamic team if you are looking to contribute to a digital health platform and make a difference in the healthcare industry.,
ACTIVELY HIRING
posted 2 weeks ago
experience14 to 20 Yrs
location
Maharashtra
skills
  • Data Integration
  • Data Migration
  • Business Intelligence
  • Artificial Intelligence
  • Cloud
  • GCP
  • AWS
  • Azure
  • ETL
  • Spark
  • Scala
  • EMR
  • Informatica
  • DataStage
  • OWB
  • Talend
  • Mongo dB
  • CouchDB
  • Cassandra
  • Cloud Storage
  • Athena
  • Glue
  • Sqoop
  • Flume
  • Hive
  • Kafka
  • Airflow
  • Presto
  • BI Reporting
  • Dashboarding
  • Tableau
  • Power BI
  • SAP BO
  • Cognos
  • Iaas
  • PaaS
  • SaaS
  • BI Reporting
  • Dashboarding
  • Data engineering
  • Data migration
  • Business intelligence
  • PreSales
  • RFP
  • RFIs
  • Data Lakes
  • Data Warehouse
  • Pyspark
  • Dataflow
  • DataProc
  • NoSQL databases
  • Graph dB
  • Big Query
  • Redshift
  • S3
  • PubSub
  • Kinesis
  • Composer
  • Spark SQL
  • EMRFS
  • Machine Learning Frameworks
  • TensorFlow
  • Pytorch
  • Looker
  • Superset
  • Containers
  • Microservices Architecture
  • Security features
  • Cloud environments
  • Business transformation projects
  • Technical advisor
  • Cloud Data related technical challenges
  • Architecture design
  • Cloud data analytics solutions
  • Data engagements
  • Data warehouse
  • Mentorship
  • Assets
  • Accelerators
Job Description
Role Overview: You would be joining Quantiphi as a Principal Architect - Data & Cloud with a focus on technical, solutioning, and analytical roles. Your role will involve architecting, designing, and implementing end-to-end data pipelines and data integration solutions for structured and unstructured data sources and targets. You will work on various data integration, ETL technologies on Cloud like Spark, Pyspark/Scala, Dataflow, DataProc, EMR, etc. You will also be responsible for architecting scalable data warehouse solutions on cloud platforms like Big Query or Redshift. Key Responsibilities: - More than 15 years of experience in Technical, Solutioning, and Analytical roles. - Experience in building and managing Data Lakes, Data Warehouse, Data Integration, Data Migration, and Business Intelligence/Artificial Intelligence solutions on Cloud platforms like GCP/AWS/Azure. - Ability to understand business requirements, translate them into functional and non-functional areas, and define non-functional boundaries. - Designing scalable data warehouse solutions on cloud platforms and working with various data integration, storage, and data pipeline tool sets. - Being a trusted technical advisor to customers and solutions for complex Cloud & Data related technical challenges. - Leading multiple data engagements on GCP Cloud for data lakes, data engineering, data migration, data warehouse, and business intelligence. - Implementing processes and systems to validate data, monitor data quality, and ensure production data accuracy and availability. - Mentoring young talent within the team and contributing to building assets and accelerators. Qualifications Required: - 14-20 years of experience in Technical, Solutioning, and Analytical roles. - Experience in architecting, designing, and implementing end-to-end data pipelines and data integration solutions. - Knowledge of Cloud platforms like GCP/AWS/Azure and various data integration, ETL technologies. - Deep knowledge of Cloud and On-Premise Databases, No-SQL databases, and BI Reporting and Dashboarding tools. - Understanding of Cloud solutions for IaaS, PaaS, SaaS, Containers, and Microservices Architecture and Design. - Experience in business transformation projects for movement of On-Premise data solutions to Clouds like GCP/AWS/Azure. - Ability to work with internal and external stakeholders to design optimized data analytics solutions. Additional Company Details: Quantiphi values a global and diverse culture built on transparency, diversity, integrity, learning, and growth. The company encourages innovation and excellence in both professional and personal life, providing a supportive and dynamic environment for its employees. Flexible, remote working options are available to foster productivity and work/life balance.,
ACTIVELY HIRING
question

Are these jobs relevant for you?

posted 3 weeks ago
experience6 to 10 Yrs
location
Chennai, Tamil Nadu
skills
  • Apache Spark
  • Big Data
  • ETL
  • Hadoop
  • Couchbase
  • Snowflake
  • HBase
  • Scala
  • Java
  • Python
  • Apache Kafka
  • Sqoop
  • Flume
  • SQL
  • Oracle
  • PLSQL
  • NoSQL
  • mongoDB
  • Kafka
  • JMS
  • MQ
  • Unix
  • Linux
  • Win
  • ETL Tools
  • Talend
  • Ab Initio
  • AWS
  • GCP
  • Agile methodology
  • Apache Hive
  • Financial industry
Job Description
You are seeking a Spark, Big Data - ETL Tech Lead for Commercial Cards Global Data Repository development team at Citigroup. You will be interacting with various teams within Citigroup and need to have exceptional communication skills across technology and business areas. As a technical lead, you will design and implement large-scale data processing pipelines using Apache Spark on the BigData Hadoop Platform. Your responsibilities will include developing and optimizing Spark applications, providing technical leadership for global software solutions, integrating data from various sources, and managing development scope, budgets, and timelines. You will also mentor and guide junior developers and stay updated with the latest trends in big data and cloud computing. Responsibilities: - Lead the design and implementation of large-scale data processing pipelines using Apache Spark on BigData Hadoop Platform. - Develop and optimize Spark applications for performance and scalability. - Integrate data from various sources, ensuring data quality and consistency. - Build and sustain relationships with senior business leaders. - Design, code, test, document, and implement application release projects. - Work with onsite development partners to ensure best practices. - Collaborate with Program Management and Quality Control teams. - Proactively communicate risks, issues, and concerns. - Ensure compliance with Citi's System Development Lifecycle and Information Security requirements. - Mentor and guide junior developers. Key Challenges: - Managing time and changing priorities in a dynamic environment. - Providing quick turnaround to software issues and management requests. - Assimilating key issues and concepts quickly. Qualifications: - Bachelor's or master's degree in computer science, Information Technology, or equivalent. - Minimum 10 years of experience in developing and managing big data solutions using Apache Spark. - Minimum 6 years of experience in leading globally distributed teams. - Strong programming skills in Scala, Java, or Python. - Hands-on experience with Apache Hive, Apache Kafka, HBase, Couchbase, Sqoop, Flume, etc. - Proficiency in SQL and experience with relational and NoSQL databases. - Demonstrated people and technical management skills. - Experience in building enterprise systems with a focus on recovery, stability, reliability, scalability, and performance. - Knowledge of data warehousing concepts and ETL processes. - Experience in performance tuning of large technical solutions. - Knowledge of data modeling, data architecture, and data integration techniques. Key Competencies: - Excellent organization skills and attention to detail. - Demonstrated sense of responsibility and capability to deliver quickly. - Excellent communication skills. - Proactive problem-solver. - Relationship builder and team player. - Negotiation, difficult conversation management, and prioritization skills. - Flexibility to handle complex projects and changing priorities. - Good analytical and business skills. - Promotes teamwork and continuous process improvement. Desirable Skills: - Experience in Java, Spring, ETL Tools like Talend, Ab Initio. - Experience/knowledge on Cloud technologies AWS, GCP. - Experience in the Financial industry. - ETL Certification, Project Management Certification. - Experience with Commercial Cards applications and processes. - Experience with Agile methodology. This job description provides a high-level overview of the work performed. Other job-related duties may be assigned as required.,
ACTIVELY HIRING
posted 1 week ago
experience14 to 20 Yrs
location
Maharashtra
skills
  • Data Integration
  • Business Intelligence
  • Artificial Intelligence
  • Cloud
  • GCP
  • AWS
  • Azure
  • ETL
  • Spark
  • Scala
  • EMR
  • Informatica
  • DataStage
  • OWB
  • Talend
  • Mongo dB
  • CouchDB
  • Cassandra
  • Cloud Storage
  • Athena
  • Glue
  • Sqoop
  • Flume
  • Hive
  • Kafka
  • Airflow
  • Presto
  • BI Reporting
  • Dashboarding
  • Tableau
  • Power BI
  • SAP BO
  • Cognos
  • Iaas
  • PaaS
  • SaaS
  • BI Reporting
  • Dashboarding
  • Sales
  • Productization
  • Data engineering
  • Data migration
  • Data quality
  • PreSales
  • RFP
  • RFIs
  • Data Lakes
  • Data Warehouse
  • Pyspark
  • Dataflow
  • DataProc
  • NoSQL databases
  • Graph dB
  • Big Query
  • Redshift
  • S3
  • PubSub
  • Kinesis
  • Composer
  • Spark SQL
  • EMRFS
  • Machine Learning Frameworks
  • TensorFlow
  • Pytorch
  • Looker
  • Superset
  • Containers
  • Microservices Architecture
  • Security features
  • Cloud environments
  • Business transformation projects
  • Technical advisor
  • Thought leader
  • Architecture design
  • Data analytics solutions
  • Stakeholders
  • Solutions Architects
  • GTM teams
  • Discovery
  • design workshops
  • Webinars
  • Tech talks
  • Feature enhancement
  • Interface with stakeholders
  • Data requirements
  • Mentorship
  • A
Job Description
As a Senior Architect - Data & Cloud at our company, you will play a crucial role in architecting, designing, and implementing end-to-end data pipelines and data integration solutions for various structured and unstructured data sources and targets. Your responsibilities will include the following: - More than 15 years of experience in Technical, Solutioning, and Analytical roles. - 5+ years of experience in building and managing Data Lakes, Data Warehouse, Data Integration, Data Migration, and Business Intelligence/Artificial Intelligence solutions on Cloud GCP/AWS/Azure. - Ability to understand business requirements and translate them into functional and non-functional areas, defining non-functional boundaries in terms of Availability, Scalability, Performance, Security, Resilience, etc. - Experience in distributed computing and enterprise environments like Hadoop, GCP/AWS/Azure Cloud. - Proficiency in various Data Integration and ETL technologies on Cloud like Spark, Pyspark/Scala, Dataflow, DataProc, EMR, etc. - Deep knowledge of Cloud and On-Premise Databases like Cloud SQL, Cloud Spanner, Big Table, RDS, Aurora, DynamoDB, Oracle, Teradata, MySQL, DB2, SQL Server, etc. - Exposure to No-SQL databases like MongoDB, CouchDB, Cassandra, GraphDB, etc. - Designing scalable data warehouse solutions on cloud on Big Query or Redshift. - Working with data integration, storage, and data pipeline tool sets like S3, Cloud Storage, Athena, Glue, Sqoop, Flume, Hive, Kafka, Pub-Sub, Kinesis, Dataflow, DataProc, Airflow, Composer, Spark SQL, Presto, EMRFS, etc. - Good understanding of Cloud solutions for IaaS, PaaS, SaaS, Containers, and Microservices Architecture and Design. - Experience in Machine Learning Frameworks like TensorFlow, PyTorch. - Knowledge of BI Reporting and Dashboarding tools like Looker, Tableau, Power BI, SAP BO, Cognos, Superset, etc. - Understanding of Security features and Policies in Cloud environments like GCP/AWS/Azure. In addition to the technical responsibilities, you will also have the opportunity to: - Lead multiple data engagements on GCP Cloud for data lakes, data engineering, data migration, data warehouse, and business intelligence. - Interface with stakeholders within IT and business to understand data requirements. - Take complete responsibility for the successful delivery of projects on parameters of Schedule, Quality, and Customer Satisfaction. - Mentor young talent within the team and contribute to building Assets and Accelerators. You will benefit from: - Working in a category-defining high-growth startup in the transformational AI, Decision Science, and Big Data Domain. - Being a part of a phenomenal growth journey and helping customers in digital transformation. - Collaborating with a diverse, proactive group of techies who are constantly raising the bar on translating data into tangible business value. - Flexible, remote working options for enhanced productivity and work-life balance.,
ACTIVELY HIRING
posted 2 days ago

Big Data Lead

Persistent Systems
experience6 to 10 Yrs
location
Karnataka
skills
  • Big Data
  • Data processing
  • Data management
  • Software development
  • Sqoop
  • Hive
  • Pig
  • Flume
  • HBase
  • Talend
  • Apache Spark
  • Java
  • Hadoop Ecosystem
  • NoSQL databases
Job Description
Role Overview: As a Big Data Lead at Persistent, you will be responsible for managing data sets that are too large for traditional database systems to handle. Your primary tasks will involve creating, designing, and implementing data processing jobs to transform data into a more usable format. Additionally, you will play a crucial role in ensuring data security and compliance with industry standards to safeguard the company's information. Key Responsibilities: - Manage customer priorities of projects and requests - Assess customer needs using a structured requirements process to prioritize business needs, advise on options, risks, and costs - Design and implement software products related to Big Data, including data models and visualizations - Participate actively within your teams and deliver solutions within tight deadlines - Proactively suggest new approaches, develop your capabilities, and contribute to the overall team improvement - Demonstrate a certain level of understanding across various technical skills, attitudes, and behaviors - Focus on delivering valuable solutions that drive business value Qualifications Required: - 6 years of experience in designing and developing enterprise application solutions for distributed systems - Understanding of Big Data Hadoop Ecosystem components such as Sqoop, Hive, Pig, and Flume - Additional experience working with Hadoop, HDFS, cluster management, Hive, Pig, MapReduce, HBase, Talend, NoSQL databases, and Apache Spark or other streaming Big Data processing (preferred) - Knowledge of Java or Big Data technologies will be a plus Please note that Persistent Ltd. fosters diversity and inclusion in the workplace, welcoming applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. The company offers a culture focused on talent development, competitive salary and benefits package, opportunity to work with cutting-edge technologies, employee engagement initiatives, annual health check-ups, and comprehensive insurance coverage for self, spouse, children, and parents. Persistent is committed to creating an inclusive environment with hybrid work options, flexible working hours, and accessible facilities for employees with physical disabilities. The company's values-driven and people-centric work environment aims to accelerate growth, impact the world positively, encourage collaborative innovation, and unlock global opportunities for employees to work and learn with the industry's best. Join Persistent to unleash your full potential at persistent.com/careers.,
ACTIVELY HIRING
posted 1 day ago
experience2 to 6 Yrs
location
Maharashtra
skills
  • Apache Spark
  • Python
  • Spark
  • Hive
  • Sqoop
  • HDFS
  • Oozie
  • Yarn
  • HBase
  • PySpark
  • Big Data ecosystem
  • Map Reduce
  • Nifi
  • Spark RDD APIs
  • Data frames
  • Datasets
  • Spark SQL
  • Spark Streaming
  • HIVE Bucketing
  • Partitioning
  • Analytical functions
  • Custom UDFs
Job Description
As a skilled PySpark Developer with 4-5 or 2-3 years of experience, you will be responsible for developing and maintaining data processing pipelines using PySpark, Apache Spark's Python API. You will collaborate closely with data engineers, data scientists, and stakeholders to design and implement scalable and efficient data processing solutions. Key Responsibilities: - Designing, developing, and maintaining PySpark data processing pipelines to handle large volumes of structured and unstructured data - Collaborating with data engineers and data scientists to understand data requirements and create efficient data models and transformations - Optimizing and tuning PySpark jobs for improved performance, scalability, and reliability - Implementing data quality checks, error handling, and monitoring mechanisms to ensure data accuracy and pipeline robustness - Developing and managing documentation for PySpark code, data pipelines, and data workflows Qualifications Required: - Bachelor's or Master's degree in Computer Science, Data Science, or a related field - Strong expertise in the Big Data ecosystem including Spark, Hive, Sqoop, HDFS, Map Reduce, Oozie, Yarn, HBase, Nifi - Experience in developing production-ready Spark applications using Spark RDD APIs, Data frames, Datasets, Spark SQL, and Spark Streaming - Strong experience in HIVE Bucketing and Partitioning, and writing complex hive queries using analytical functions - Knowledge of writing custom UDFs in Hive to support custom business requirements If you meet the qualifications mentioned above and are interested in this position, please email your resume to careers@cdslindia.com, mentioning the position applied for in the subject column.,
ACTIVELY HIRING
posted 0 days ago
experience6 to 10 Yrs
location
Karnataka
skills
  • MapReduce
  • Hive
  • Sqoop
  • Spark
  • Storm
  • Kafka
  • Pig
  • Athena
  • Glue
  • Snowflake
  • EMR
  • HBase
  • Cassandra
  • MongoDB
  • Flume
  • Kafka
  • RabbitMQ
  • Big Data Hadoop
  • SparkStreaming
  • Lambda Architecture
Job Description
Role Overview: You will work on collecting, storing, processing, and analyzing large sets of data. Your main focus will be on selecting the best solutions for these tasks, as well as maintaining, implementing, and monitoring them. Additionally, you will be responsible for integrating these solutions with the existing company architecture. Key Responsibilities: - Utilize your 6+ years" experience in Big Data technologies such as Hadoop, MapReduce, Hive, Sqoop, and Spark to design and implement high data volume solutions for ETL & Streaming purposes. - Build stream-processing systems using tools like Storm, Spark-Streaming, and Kafka streams. - Work with Big Data tools like Pig, Hive, Athena, Glue, Snowflake, and EMR to handle data effectively. - Utilize NoSQL databases including HBase, Cassandra, and MongoDB. - Implement various ETL techniques and frameworks, such as Flume. - Use messaging systems like Kafka or RabbitMQ for data processing. - Understand Lambda Architecture and its advantages and disadvantages. Qualifications Required: - 6+ years of experience in Big Data technologies (Hadoop, MapReduce, Hive, Sqoop, Spark) - Hands-on expertise in designing and implementing high data volume solutions for ETL & Streaming - Experience with stream-processing systems (Storm, Spark-Streaming, Kafka streams) - Proficiency in working with Big Data tools (Pig, Hive, Athena, Glue, Snowflake, EMR) - Familiarity with NoSQL databases (HBase, Cassandra, MongoDB) - Knowledge of ETL techniques and frameworks (Flume) - Experience with messaging systems (Kafka, RabbitMQ) - Good understanding of Lambda Architecture (Note: No additional details about the company were provided in the job description.),
ACTIVELY HIRING
posted 1 week ago
experience5 to 9 Yrs
location
Maharashtra
skills
  • Python
  • Scala
  • Hadoop
  • Impala
  • Hive
  • Sqoop
  • Spark
  • SQL
  • ETL
  • Data Streaming
Job Description
Role Overview: You will be a Data Analyst in the Business Intelligence Unit (BIU) based in Mumbai. Your primary responsibility will be to develop applications using Big data tools and the Hadoop platform. Additionally, you will engage in discussions with stakeholders and manage deliverables. Key Responsibilities: - Design solutions using Hadoop based technologies - Ingest data from files, RDBMS, and streams. Process the data with Hadoop, Python, and Spark - Develop programs/algorithms in Python for data cleaning and processing - Write efficient code using Big Data tools - Implement scalable solutions to handle increasing data volumes using big data technologies - Understand the working of developed use cases - Engage in discussions with stakeholders Qualifications: Essential: - BE/BTECH Desired: - BE/BTECH Experience: Essential: - 5 years Desired: - 5-7 years Desired Skills and Competencies: - Proficiency in Python/Scala - Good knowledge of Hadoop architecture - In-depth experience in Impala, Hive, Sqoop, Spark, any ETL tool, and comfortable in writing SQL queries - Banking Domain Knowledge preferred - Hands-on experience in Data Streaming (Good to have) - Certifications related to Hadoop technologies (Good to have) - Team player with good interpersonal skills - Able to interact with business users and work independently (Note: No additional details about the company were provided in the job description.),
ACTIVELY HIRING
posted 1 month ago

ENGINEER - DATABASE SERVICES

Times Internet for Marketers
experience5 to 10 Yrs
location
Noida, Uttar Pradesh
skills
  • MongoDB
  • AWS
  • SQL
  • NoSQL
  • Replication
  • Scalability
  • Performance tuning
  • Hadoop
  • HDFS
  • Hive
  • Flume
  • Sqoop
  • Spark
  • MySQLMariaDB
  • RedisDB
  • RDS
Job Description
As a MySql Database Admin at Times Internet, your role will involve participating in the design, implementation, automation, optimization, and ongoing operational administration tasks for backend systems running on MySQL/MariaDB/MongoDB/RedisDB/Hadoop database infrastructure. You should be prepared to handle instant support and operational challenges related to infrastructure platforms running database services. Key Responsibilities: - Hands-on experience with MySQL/MariaDB RDBMS and related tools such as Percona Xtrabackup and Percona-tool-kit. - Proficiency in MongoDB NoSQL and Cache (RedisDB) datastores. - Experience working on both Private and Public IT Infrastructure clouds like AWS. - Optimization of database SQL or No-SQL queries. - Implementation of best optimized and secure practices for RDS, NoSQL, and Cache database stores. - Infra-resource planning, database upgrades/patching, backup/recovery, and database troubleshooting. - Handling database support activities including Replication, Scalability, Availability, Performance tuning/optimizations on servers managing large data volumes. - Designing highly scalable schema design for applications using No-SQL, RDBMS, and Cache databases. - Good to have experience with Hadoop Technologies like HDFS, Hive, Flume, Sqoop, Spark, etc. Qualifications Required: - 5-10 years of experience in database administration. - Strong expertise in MySQL/MariaDB, MongoDB, RedisDB, and Hadoop technologies. - Experience in optimizing and securing database queries and infrastructures. - Proficiency in cloud platforms like AWS and hands-on experience with database support activities. - Excellent problem-solving skills and ability to handle operational challenges effectively. Join Times Internet, the largest digital products company in India, and be a part of a dynamic team driving the Internet revolution in the country.,
ACTIVELY HIRING
posted 1 month ago

Data Technical Lead

SMC Squared India
experience7 to 11 Yrs
location
All India
skills
  • Scala
  • Python
  • SQL
  • HIVE
  • HBase
  • Impala
  • Sqoop
  • Kafka
  • Flume
  • Airflow
  • Jenkins
  • Bamboo
  • Github
  • Bitbucket
  • AzureAWS Databricks
  • PySpark
  • Parquet
  • Nexus
Job Description
As a Data Technical Lead at SMC Squared, you will be responsible for managing the day-to-day operations of the Data Platform in Azure/AWS Databricks NCS. Your role involves overseeing the development, enhancement, support, and maintenance of data availability, data quality, performance enhancement, and system stability. You will design and implement data ingestion pipelines, ensure smooth and efficient data pipelines, and adhere to security, regulatory, and audit control guidelines. Additionally, you will drive optimization, continuous improvement, and efficiency in the data platform. **Roles & Responsibilities:** - Design and implement data ingestion pipelines using Azure Databricks - Ensure smooth and efficient data pipelines operation - Develop scalable and re-usable frameworks for ingesting data sets - Integrate end-to-end data pipelines to maintain data quality and consistency - Work with event-based/streaming technologies for data ingestion and processing - Collaborate with project team members to support delivery of project components - Evaluate tools against customer requirements - Provide technical advice and resolution on Cloud and Databricks - Provide on-call and after-hours/weekend support - Fulfill Service Requests related to Data Analytics platform - Lead optimization and continuous improvement initiatives - Conduct technical reviews as part of release management - Adhere to data security standards and controls - Lead the design, development, and deployment of advanced data pipelines - Collaborate with stakeholders to build end-to-end data solutions - Ensure adherence to data governance, security, and compliance - Mentor a team of data engineers - Implement CI/CD practices for data engineering pipelines **Years Of Experience:** - The ideal candidate will have 8+ years of professional experience **Educational Qualification & Certifications (Optional):** - Bachelors degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent **Skill Set Required:** - Minimum seven plus years of experience in data analytics field - Experience with Azure/AWS Databricks - Experience in building and optimizing data pipelines, architectures, and data sets - Proficiency in Scala or Python, PySpark, and SQL - Ability to troubleshoot and optimize complex queries on the Spark platform - Knowledge of structured and unstructured data design, data access, and data storage techniques - Expertise in designing and deploying data applications on cloud solutions - Hands-on experience in performance tuning and optimizing code in Databricks environment - Analytical and problem-solving skills in a big data environment - Familiarity with various technologies including HIVE, HBase, Impala, Parquet, Sqoop, Kafka, Flume, Airflow, Jenkins, Bamboo, Github, Bitbucket, Nexus About SMC Squared: SMC Squared is focused on accelerating digital transformation for leading US and multinational companies by building and managing Global Capability Centres. As a part of Hexaware Technologies, we deliver enhanced digital capabilities and automation platforms while prioritizing people-centric transformation. EEO Statement: SMC Squared is an equal opportunity employer welcoming applications from candidates of diverse backgrounds and experiences. If you are passionate about design and contributing to impactful projects, we invite you to apply for this exciting opportunity.,
ACTIVELY HIRING
posted 3 weeks ago
experience5 to 9 Yrs
location
Gujarat, Vadodara
skills
  • SQL
  • Sqoop
  • Hadoop
  • Microsoft Excel
  • Python
  • Scala
  • Java
  • TSQL
  • PLSQL
  • Git
  • Jenkins
  • Autosys
  • Oozie
  • Azure Data Factory
  • Azure Data bricks Spark
  • PySpark
  • Azure Data Engineer
  • DATA Ingestion
  • Curation
  • Semantic Modelling
  • Optimization of data model
  • Rahona
Job Description
Role Overview: As a Production Specialist at Wipro Limited, your role is to support process delivery by ensuring daily performance of the Production Specialists, resolving technical escalations, and developing technical capability within the team. You will be responsible for overseeing and supporting the process, reviewing daily transactions on performance parameters, and supporting the team in improving performance parameters by providing technical support and process guidance. Additionally, you will handle technical escalations through effective diagnosis and troubleshooting of client queries, manage and resolve technical roadblocks/escalations, and provide product support and resolution to clients. Key Responsibilities: - Oversee and support process by reviewing daily transactions on performance parameters - Review performance dashboard and scores for the team - Support the team in improving performance parameters by providing technical support and process guidance - Record, track, and document all queries received, problem-solving steps taken, and total successful and unsuccessful resolutions - Ensure standard processes and procedures are followed to resolve all client queries - Resolve client queries as per the SLAs defined in the contract - Develop understanding of process/product for team members to facilitate better client interaction and troubleshooting - Document and analyze call logs to spot most occurring trends to prevent future problems - Identify red flags and escalate serious client issues to Team Leader in cases of untimely resolution - Ensure all product information and disclosures are given to clients before and after the call/email requests - Handle technical escalations through effective diagnosis and troubleshooting of client queries - Manage and resolve technical roadblocks/escalations as per SLA and quality requirements - Timely escalate issues to TA & SES if unable to resolve - Provide product support and resolution to clients by performing question diagnosis while guiding users through step-by-step solutions - Troubleshoot all client queries in a user-friendly, courteous, and professional manner - Offer alternative solutions to clients with the objective of retaining customers and clients business - Organize ideas and effectively communicate oral messages appropriate to listeners and situations - Follow up and make scheduled call backs to customers to record feedback and ensure compliance to contract SLAs Qualifications Required: - Cloud certified in Azure Data Engineer - Proficiency in Azure Data Factory, Azure Data bricks Spark (PySpark or Scala), SQL, DATA Ingestion, and Curation - Experience in Azure ingestion from on-prem source (e.g., mainframe, SQL server, Oracle) and Sqoop/Hadoop - Strong programming skills in Python, Scala, or Java - Strong SQL skills (T-SQL or PL-SQL) - Experience with data files movement via mailbox and source-code versioning/promotion tools like Git/Jenkins - Knowledge of orchestration tools such as Autosys, Oozie - Experience working with mainframe files and in an Agile environment using JIRA/Confluence tool Company Additional Details: Wipro Limited is a leading technology services and consulting company dedicated to building innovative solutions that address clients" most complex digital transformation needs. With over 230,000 employees and business partners across 65 countries, Wipro helps clients realize their boldest ambitions and build future-ready, sustainable businesses. Join a business powered by purpose and be a part of a place that empowers you to design your own reinvention. Realize your ambitions at Wipro. Applications from people with disabilities are explicitly welcome.,
ACTIVELY HIRING
posted 1 week ago
experience3 to 7 Yrs
location
Karnataka
skills
  • Scala
  • Spark
  • Python
  • Hadoop
  • HDFS
  • Hive
  • Sqoop
  • Kafka
  • Oracle
  • Git
  • Jenkins
  • Artifactory
  • Tableau
  • Hudi
  • Parquet
  • Apache Nifi
  • MSSQL
  • AtScale
Job Description
As a qualified candidate for this role, you should have a Bachelors Degree in Computer Science, Computer Engineering, or a related technical field. A Masters Degree or other advanced degree would be preferred. You should have 4-6+ years of total experience with at least 2+ years of relevant experience in Big Data platforms. Your strong analytical, problem-solving, and communication skills will be essential for success in this role. Key Responsibilities: - Possess 3+ years of experience working with big data and the Hadoop ecosystem, including Spark, HDFS, Hive, Sqoop, Hudi, Parquet, Apache Nifi, and Kafka. - Demonstrated hands-on experience in Scala/Spark, with Python skills considered a plus. - Proficiency in Oracle and MS-SQL databases is required. - Familiarity with job schedulers like CA or AutoSys is preferred. - Experience with source code control systems such as Git, Jenkins, and Artifactory is necessary. - Exposure to platforms like Tableau and AtScale would be beneficial. Qualifications Required: - Bachelors Degree in Computer Science, Computer Engineering, or a related technical field. - Masters Degree or other advanced degree preferred. - 4-6+ years of total experience with 2+ years in relevant Big Data platforms. - Strong analytical, problem-solving, and communication/articulation skills. - Hands-on experience with Scala/Spark and Python (preferred). - Knowledge of Oracle, MS-SQL databases, job schedulers, and source code control systems. - Familiarity with Tableau and AtScale platforms is a plus.,
ACTIVELY HIRING
posted 2 months ago
experience6 to 10 Yrs
location
Noida, Uttar Pradesh
skills
  • Hadoop
  • HDFS
  • Kafka
  • Flume
  • Hive
  • Pig
  • Sqoop
  • HBase
  • Cassandra
  • Neo4j
  • MongoDB
  • Kibana
  • Tableau
  • Pentaho
  • Elastic Search
  • Google Analytics
  • Kerberos
  • MR
  • Spark Streaming
  • Spark SQL
  • Spark ML
  • Apache NiFi
  • Hortonworks Data Platform
  • D3js
  • Zeppelin
  • Grafana
  • Scrapy
  • Open LDAP
  • Knox
  • Ranger
Job Description
As an experienced professional with over 7+ years of experience in Noida, your role will involve the following key responsibilities: - Reviewing and understanding business requirements to ensure timely completion of development tasks with minimal defects - Collaborating with a software development team to implement best practices and enhance the performance of Data applications for meeting client needs - Engaging with various teams and customers to translate their business challenges into innovative solutions - Conducting research on new Big Data technologies to align them with the business and technology strategy - Operating within a rapid and agile development process to accelerate speed to market while maintaining necessary controls To qualify for this role, you should possess the following qualifications: - BE/B.Tech/MCA degree with a minimum of 6+ years of overall IT experience, including 4+ years of hands-on experience in design and development using the Hadoop technology stack and programming languages - Proficiency in at least 2 or more areas such as Hadoop, HDFS, MR, Spark Streaming, Spark SQL, Spark ML, Kafka/Flume, Apache NiFi, Hortonworks Data Platform, Hive/Pig/Sqoop, NoSQL Databases (HBase/Cassandra/Neo4j/MongoDB), Visualization & Reporting frameworks (D3.js, Zeppelin, Grafana, Kibana, Tableau, Pentaho), Scrapy, Elastic Search, Google Analytics data streaming, Data security (Kerberos/Open LDAP/Knox/Ranger) - Strong knowledge of the current technology landscape and ability to anticipate industry trends - Familiarity with Big Data integration with Metadata Management, Data Quality, and Master Data Management solutions, structured/unstructured data - Active involvement in the community through articles, blogs, or speaking engagements at conferences This job offers you an opportunity to work on cutting-edge technologies and be part of a dynamic team focused on delivering innovative solutions to address business challenges.,
ACTIVELY HIRING
posted 2 months ago

GCP Lead

Impetus
experience4 to 11 Yrs
location
All India
skills
  • GCP
  • Hadoop
  • NoSQL
  • Spark
  • Kafka
  • Python
  • Java
  • Big Data technologies
  • Google cloud storage
  • Google compute engine
  • Cloud SQL
  • Cloud IAM
Job Description
As an experienced IT professional with 8-11 years of experience, you will be required to have a BE/B.Tech/MCA/MS-IT/M.Tech or any other Engineering degrees in related fields. Your role will involve extensive production experience (5 Years) with GCP, with other cloud experience being a strong bonus. Additionally, you should have a strong background in Data engineering with 4-5 years of experience in Big Data technologies including Hadoop, NoSQL, Spark, Kafka, etc. Exposure to enterprise application development is also a must for this role. Your key responsibilities will include: - Effectively using GCP managed services such as Dataproc, Dataflow, pub/sub, Cloud functions, Cloud composer, Big Query, Big Table (at least 4 of these services). - Demonstrating strong experience in Big Data technologies like Hadoop, Sqoop, Hive, and Spark, including DevOPs. - Having good hands-on expertise in either Python or Java programming. - Understanding GCP core services like Google cloud storage, Google compute engine, Cloud SQL, Cloud IAM. - Having knowledge on GCP services like App engine, GKE, Cloud Run, Cloud Built, Anthos. - Driving the deployment of customers" workloads into GCP, providing guidance, cloud adoption model, service integrations, appropriate recommendations, and technical roadmaps for GCP cloud implementations. - Architecting and designing technical solutions based on industry standards using GCP - IaaS, PaaS, and SaaS capabilities. - Designing technology components for enterprise solutions and defining solution architectures and reference architectures with a focus on cloud technologies. - Acting as a subject-matter expert or developer around GCP and becoming a trusted advisor to multiple teams. - Coaching and mentoring engineers to raise the technical ability of the rest of the team or to become certified in required GCP technical certifications. If there are any additional details about the company in the job description, they will be included here.,
ACTIVELY HIRING
logo

@ 2025 Shine.com | All Right Reserved

Connect with us:
  • LinkedIn
  • Instagram
  • Facebook
  • YouTube
  • Twitter