We at Coforge are Sr. Data Engineer with the following skillset:
Responsibilities:
A Sr. Data Engineer is proficient in the development of all aspects of data processing including data warehouse architecture/modeling and ETL processing. The position focuses on development and delivery of analytical solutions using various tools including AWS Glue, Coalesce, Airflow, Snowflake and AWS.
Data Engineer must be able to work autonomously with little guidance or instruction to deliver business value.
Qualifications Education bachelor’s degree in computer science or MIS related area required or equivalent experience (industry experience substitutable).
6-8 years of total experience in Data engineering/Cloud development activity.
1+ years of experience in Banking and financial domain is Nice to have.
Must be extremely proficient in Data Warehouse ETL Design/Architecture, dimensional/relational data modelling.
Experience in at least one ETL development project, writing/analyzing complex stored procedures.
Should have entry level/intermediate experience in Python/Py Spark – working knowledge on spark/pandas data frame, spark multi-threading, exception handling, familiarity with different boto3 libraries, data transformation and ingestion methods, ability to write UDF.
Snowflake – Familiarity with stages and external tables, commands in snowflake like copy, unload data to/from S3, working knowledge of variant data type, flattening nested structure thru SQL, familiarity with marketplace integrations, role-based masking, pipes, data cloning, logs, user and role management is nice to have.
Familiarity with Coalesce/dbt is an added advantage for this job.
Collibra integration experience for Data Quality and Governance in ETL.
AWS – Should have hands-on experience with S3, Glue (jobs, triggers, workflow, catalog, connectors, crawlers), CloudWatch, RDS and secrets manager.
AWS - VPC, IAM, Lambda, SNS, SQS, MWAA is nice to have.
Should have hands-on experience with version controlling tools like GitHub, working knowledge on configuring, setting up CI/CD pipelines using yaml, pip files.
Streaming Services – Familiarity with Confluent Kafka or spark streaming, or Kinesis (or equivalent) is nice to have.
Data Vault 2.0 (hubs satellite links) experience will be a PLUS. Interpersonal.
Highly proficient in Publisher, PowerPoint, SharePoint, Visio, Confluence and Azure DevOps.
Working knowledge of best practices in value-driven development (requirements management, prototyping, hypothesis-driven.