Satyanarayana Annepogu，加拿大安大略省多伦多的开发者
Satyanarayana is available for hire
Hire Satyanarayana

Satyanarayana Annepogu

Verified Expert  in Engineering

Database Developer

Location
Toronto, ON, Canada
Toptal Member Since
October 25, 2022

Satya is a senior data engineer with over 15 years of IT experience designing and developing data warehouses for banking and insurance clients. He specializes in designing and building modern data pipelines and streams using AWS and Azure Data engineering stack. Satya is an expert in delivering modernization of enterprise data solutions using AWS and Azure cloud data technologies.

InformaticaData EngineeringBig DataData AnalysisData AnalyticsData Modeling大数据架构Amazon RDSMessage QueuesData VisualizationData IntegrationSQLETLSQL存储过程OracleDimensional ModelingTableauOrchestrationBusiness IntelligenceCSVDBT (Data Build Tool)

Portfolio

Millicom
数据工程，亚马逊网络服务，大数据，Spark, SQL, Python...
海姆斯登服务公司
Azure数据工厂，数据工程，数据管道，SQL...
IBM
PySpark, Azure, Azure数据工厂，Microsoft Power BI, Spark, Spark SQL...

Experience

数据工程- 7年Python - 5 yearsPySpark - 4 years亚马逊网络服务(AWS)——3年Azure - 3 yearsAzure Synapse Analytics - 3年Microsoft Power BI - 3年Snowflake - 1 year

Availability

Full-time

Preferred Environment

Data Engineering, 亚马逊网络服务(AWS), Azure, Databricks, Python, PySpark, Hadoop, Snowflake, Data Warehousing, ETL Tools

The most amazing...

...我做过的项目是设计, developing, 并支持基于云的和传统的数据仓库应用程序.

Work Experience

Data Engineer

2023 - 2024
Millicom
  • 领导实施用于自动化ETL流程的AWS Glue, reducing data processing time and improving data accuracy for telecom network performance data, 客户交互, 以及账单信息.
  • 利用AWS Lambda功能开发无服务器数据管道, 促进CRM系统之间的无缝集成, 网络基础设施, IoT devices, 以及电信生态系统内的外部资源.
  • Architected solutions using Amazon S3 (AWS S3) to optimize data storage and retrieval, implementing cost-effective and scalable data lakes to accommodate large volumes of network performance data, 客户交互, 以及运营指标.
  • 使用AWS步骤函数编排复杂的工作流, ensuring efficient coordination and execution of multi-step data processing tasks for proactive network health monitoring and dynamic service provisioning.
  • 利用Amazon Redshift作为数据仓库解决方案, enabling high-performance analytical queries to derive actionable insights into network performance, customer behavior, 服务使用模式.
  • 集成AWS数据管道，实现数据移动和转换的自动化, streamlining operational processes and enhancing data availability for real-time decision-making and strategic planning.
  • Implemented robust security measures using AWS Identity and Access Management (IAM) and Amazon VPC, ensuring data privacy and regulatory compliance for sensitive network performance data, 客户交互, 以及账单信息.
  • 利用AWS Lambda函数创建无服务器数据管道, ensuring seamless integration between disparate systems and services within the BN Bank Norway ecosystem.
  • Optimized data storage and retrieval by architecting solutions using Amazon S3, implementing cost-effective and scalable data lakes to accommodate the bank's growing data volumes.
Technologies: 数据工程，亚马逊网络服务，大数据，Spark, SQL, Python, Scala, AWS Lambda, AWS Glue, Amazon S3 (AWS S3), Data Transformation, 大数据架构, Amazon RDS, Message Queues, Redshift, Amazon Athena, Amazon Elastic MapReduce (EMR), ETL Tools, Spark SQL, APIs, Azure Databricks, Data Pipelines, Apache Kafka, Data Integration, ETL, Data Processing, Data, Data Analysis, Data Analytics, Data Visualization, Large-scale Projects, Teamwork, Data Modeling, ELT, Apache Airflow, Terraform

具有Azure数据工厂专业知识的数据分析师

2022 - 2023
海姆斯登服务公司
  • Designed and Developed data Ingestion pipelines using ADF and processing layer using Databricks Notebooks with pySpark.
  • Lead in the planning, design, development, testing, implementation, documentation, 以及数据管道的支持.
  • 使用ADF暂停和恢复Azure SQL数据仓库. 以业务规则用例作为可重用资产的各种ADF管道.
  • 用于存储连接字符串详细信息的Azure密钥库, certificates and used the key vaults in Azure Data factory while creating linked services. 管道的编排和自动化.
  • 执行SCD2, SCD1，每日，每周和每月批次. POC与Apache Spark使用pySpark, Spark-SQL用于各种复杂的数据转换需求.
Technologies: Azure数据工厂，数据工程，数据管道，SQL, Microsoft Dynamics 365, ETL Tools, Spark, Spark SQL, APIs, Azure Databricks, Big Data, Apache Kafka, AWS Lambda, AWS Glue, Amazon S3 (AWS S3), Data Transformation, 大数据架构, Amazon RDS, Message Queues, Redshift, Amazon Athena, Data Integration, ETL, Data Processing, Data, Data Analysis, Data Analytics, Data Visualization, Large-scale Projects, Microsoft SQL Server, Teamwork, Data Modeling, ELT, Apache Airflow, 数据构建工具(dbt), Azure Blob存储API, Azure Data Lake, Azure Synapse

数据工程师(Azure)和技术主管

2021 - 2022
IBM
  • Designed and Developed data Ingestion pipelines using ADF and processing layer using Databricks Notebooks with pySpark, Lead in the planning, design, development, testing, implementation, documentation, 以及数据管道的支持.
  • 使用ADF暂停和恢复Azure SQL数据仓库. 以业务规则用例作为可重用资产的各种ADF管道. 摄取csv，固定宽度，excel文件.
  • 用于存储连接字符串详细信息的Azure密钥库, certificates and used the key vaults in Azure Data factory while creating linked services. 使用Web活动的自动管道故障电子邮件通知.
  • 管道的编排和自动化. POC与Apache Spark使用pySpark, Spark-SQL用于各种复杂的数据转换需求. 用于自动化管道的PowerShell脚本.
  • Collaborated with ETL teams both Client and IBM and Analyzed On- Prem Informatica based ETL solutions and designed ETL solution using Azure Data Factory Pipelines and Azure Databricks PySpark and Spark-SQL.
  • Azure数据工厂、Azure DataBricks中管道的性能调优.
技术:PySpark, Azure, Azure数据工厂，Microsoft Power BI, Spark, Spark SQL, Azure Databricks, Python, APIs, ETL Tools, Data Engineering, Azure Synapse Analytics, Data Pipelines, SQL, Big Data, Apache Kafka, AWS Lambda, AWS Glue, Amazon S3 (AWS S3), Data Transformation, 大数据架构, Redshift, Amazon Athena, Data Integration, Financial Services, Technical Leadership, ETL, Data Processing, Data, Data Analysis, Data Analytics, Data Visualization, Large-scale Projects, Microsoft SQL Server, Teamwork, 数据库体系结构, Data Modeling, ELT, Terraform, 数据构建工具(dbt), Excel VBA, Azure Blob存储API, Azure Data Lake, Azure Synapse

Team Lead, Sr. ETL Consultant

2014 - 2018
IBM India
  • Developed solution in highly demanding environment and provide hands on guidance to other team members. 负责复杂的ETL需求和设计. Implemented Informatica based ETL solution fulfilling stringent performance requirements.
  • Collaborated with product development teams and senior designers to develop architectural requirements to ensure client satisfaction with product.
  • 评估需求的完整性和准确性. 确定ETL团队的需求是否可行. Conducted impact assessment and determine size of effort based on requirements.
  • Developed full SDLC project plans to implement ETL solution and identify resource requirements. Performed as active, leading role in shaping and enhancing overall ETL Informatica architecture.
  • Identify, recommend, and implement ETL process and architecture improvements. Assist and verify design of solution and production of all design phase deliverable.
  • Managed build phase and quality assure code to ensure fulfilling requirements and adhering to ETL architecture. 解决困难的设计和开发问题. 为团队提供项目目标的愿景.
  • 确保讨论和决定导致结束. 保持健康的群体动力. Assure that team addresses all relevant issues within specifications and various standards.
  • 成功地使团队熟悉客户需求, specifications, design targets, development process, design standards, techniques, 以及支持任务执行的工具.
技术:Informatica, PL/SQL, PL/SQL Tuning, Netezza, Unix Shell Scripting, ETL Tools, Data Engineering, Azure Synapse Analytics, Microsoft Power BI, Spark SQL, APIs, Azure Databricks, SQL, Big Data, Apache Kafka, AWS Lambda, Data Transformation, Data Integration, Financial Services, Technical Leadership, ETL, Data Processing, Data, Data Analysis, Data Analytics, Data Visualization, Microsoft SQL Server, Teamwork, 数据库体系结构, Data Modeling, ELT, Apache Airflow, Terraform, Excel VBA, Azure Blob存储API, Azure Data Lake, Azure Synapse

Sr. Informatica Designer

2009 - 2014
IBM荷兰-阿佩尔多伦和IBM印度-海得拉巴
  • 在与建模者的功能知识转移会议的标题. 领导技术设计会议，设计各个层. Analyzing functional design document and prepared analysis sheets for individual layers.
  • Extensively working on technical design generation set of documents and amended as suitable for present release.
  • 100% Transition sign off for all 4 releases Ramp up post transition Successful delivery of projects during steady state Process improvements and suggestions. 参与所有4个迭代的交叉训练资源.
  • Identified training needs for teams and organized for training to fill knowledge gap. Received laurels and appreciation both from client and IBM Received monetary and monumental/certifications awards.
技术:Informatica, IBM Db2, Oracle, Unix Shell Scripting, Autosys, Cognos TM1, ETL Tools, Unix, SQL, Data Transformation, Data Integration, Financial Services, ETL, Data Processing, Data, Data Analysis, Data Analytics, Teamwork, 数据库体系结构, ELT, Excel VBA

Senior ETL Developer

2008 - 2009
Genisys集成系统公司.Ltd., Bangalore, India
  • Developed mapping for type 2 dimension for updating already existing rows and inserting new rows in targets. Developed Actuate reports like Drill-Up and Drill-Down reports, Series reports and Parallel reports.
  • 在Actuate工作，用于格式化与不同流程相关的报告. 开发与生成相关的仪表板, failed, 等待报告和计划报告，以一刻钟为单位, hour, day, month, and year.
  • Analyzed number of reports generated, failed, scheduled, and waiting and developed for above reports.
  • Developed mapping for type 2 dimension for updating already existing rows and inserting new rows in targets.
技术:Informatica, Oracle, Unix Shell Scripting, ETL Tools, Unix, SQL, Data Transformation, Data Integration, ETL, Data Processing, Data, ELT

Senior ETL Developer

2007 - 2008
Magna Infotech Pvt.Ltd., Bangalore, India
  • Developed mapping for type 2 dimension for updating already existing rows and inserting new rows in targets. 在Actuate工作，用于格式化与不同流程相关的报告.
  • Developed Actuate reports like Drill-Up and Drill-Down reports, Series reports and Parallel reports. Analyzed number of reports generated, failed, scheduled, and waiting and developed for above reports.
  • 开发与生成相关的仪表板, failed, 等待报告和计划报告，以一刻钟为单位, hour, day, month, and year.
  • Developed mapping for type 2 dimension for updating already existing rows and inserting new rows in targets.
  • 有从三维建模到ETL设计的经验.
技术:Informatica, Unix Shell Scripting, Oracle, ETL Tools, Unix, SQL, Data Transformation, Data Integration, ETL, Data Processing, Data, Retail & Wholesale, ELT

ETL Lead Developer

2005 - 2007
TechnoSpine Solutions，班加罗尔，印度
  • Developed mapping for type 2 dimension for updating already existing rows and inserting new rows in targets 在Actuate工作，用于格式化与不同流程相关的报告.
  • Developed Actuate reports like Drill-Up and Drill-Down reports, Series reports and Parallel reports. Analyzed number of reports generated, failed, scheduled, and waiting and developed for above reports.
  • 开发与生成相关的仪表板, failed, 等待报告和计划报告，以一刻钟为单位, hour, day, month, and year.
  • Developed mapping for type 2 dimension for updating already existing rows and inserting new rows in targets. 有从三维建模到ETL设计的经验 Certifications and Accomplishments.
技术:Informatica, Oracle, Autosys, ETL Tools, Unix, SQL, Data Transformation, Data Integration, ETL, Data Processing, Data, Retail & Wholesale, ELT

数据工程师(Azure)和技术主管:

Tool Client Rate Desk is a web-based tool providing authoritative cash management pricing arrangements and contract information for the mid-market and large corporate client segments. 该应用程序由业务联络中心使用, 客户关系经理, 现金管理及销售人员.

Data Engineer Azure

优化ETL管道，减少处理时间和成本.
在Azure Databricks中实现实时分析，以获得可操作的见解.
与Azure数据服务无缝集成.
建立健壮的数据治理和遵从性措施.
增强数据处理工作流的性能.

Languages

Python, SQL, Snowflake, Excel VBA, Scala

Frameworks

Spark, Hadoop

Libraries/APIs

PySpark, Azure Blob存储API

Tools

Autosys, Microsoft Power BI, Spark SQL, AWS Glue, Amazon Athena, Apache Airflow, Amazon Elastic MapReduce (EMR), Terraform

Paradigms

ETL

Platforms

亚马逊网络服务(AWS), Azure, Databricks, Oracle, Unix, Azure Synapse Analytics, AWS Lambda, Azure Synapse, Apache Kafka

Storage

SQL存储过程, Data Pipelines, Amazon S3 (AWS S3), Redshift, Data Integration, Microsoft SQL Server, 数据库体系结构, IBM Db2, PL/SQL, Netezza

Other

Data Engineering, Data Warehousing, ETL Tools, Informatica, Azure Data Factory, Azure Databricks, APIs, Big Data, Data Transformation, 大数据架构, Amazon RDS, Message Queues, Financial Services, Technical Leadership, Data Processing, Data, Data Analysis, Data Analytics, Data Visualization, Large-scale Projects, Teamwork, Data Modeling, ELT, Microsoft Dynamics 365, 数据构建工具(dbt), Azure Data Lake, Unix Shell Scripting, Cognos TM1, PL/SQL Tuning, Azure数据湖分析

Industry Expertise

Retail & Wholesale

1998 - 2002

技术或电气工程学士学位

贾瓦哈拉尔尼赫鲁理工大学-海德拉巴，印度

2023年9月至今

AWS认证云从业者

Amazon Web Services

2021年12月至今

Azure Data Engineer

Microsoft

有效的合作

如何使用Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

开始你的无风险人才试验

与你选择的人才一起工作，试用最多两周. 只有当你决定雇佣他们时才付钱.

对顶尖人才的需求很大.

Start hiring