Start date
May 2023
Duration
3 months
Enrol now
Book your seat
Course Fee
35k / Module
Study Mode
Online / On Campus
Program overview:
‘Data Engineering Foundation’ is a tailored but specialized course developed to assist potential Students and developers to develop their potential for Data Science and Data Engineering. In this course, you’ll discover how to build an effective data architecture, streamline data processing, and maintain large-scale data systems. In addition to working with Python, you’ll also grow your language skills as you work with SQL, and other tools, to create Data Engineering Foundation pipelines, converging in comprehensive understanding and execution of Data Science, automate common file system tasks, and build a high-performance database.
Program overview:
‘Data Engineering’ is a tailored but specialized course developed to assist potential Students and developers to develop their potential for Data Engineering. In this course, you’ll discover how to build an effective data architecture, streamline data processing, and maintain large-scale data systems. In addition to working with Python, you’ll also grow your language skills as you work with Shell, SQL, and other languages, to create data engineering pipelines, automate common file system tasks, and build a high-performance database.
- Students from CS, IT or Software background can join.
- anyone who wants to learn this diverse field.
- mprove Programming skills
- How to deploy application on multiple clouds
- Data Acquisition from from different platforms
Who should attend:
- Students from CS, IT or Software background can join.
- anyone who wants to learn this diverse field.
Objectives:
- Imbibe Python Programming Skills
- Investigating and perceiving datasets
- To analyze and interpret any given data set.
- What outcomes may be tabulated from data sets
Learning Outcome:
On successful completion of this program, participants will be able to:Knowledge
- Working knowledge of creation of Tables and Data Sets
- Learn critical concepts around data sets and manipulations techniques.
- Understanding different data storage options such as SQL.
- Learn application scalability in terms of deployment.
- Viewpoints and tips from Data professionals on starting a career in this domain.
- Identifying patterns and data used for descriptive and predictive analytics.
Skills
- Python programming basics including data structures, logic, working with files, invoking APIs, using libraries such as Pandas, NumPy, and Seaborn.
- SQL query language, SELECT, INSERT, UPDATE, DELETE statements, database functions, stored procs, working with multiple tables, Joins, & transactions.
- Schedule, automate, and monitor data pipelines. Run data quality checks, track data lineage, and work with data pipelines in production.
- Identifying matrixes important to stakeholders and creating dashboards to represent.
- Creating and hypothecating inferences to be. Highlighting data in the form of descriptive statistics.
Knowledge
- Working knowledge of Data Engineering Ecosystem and Lifecycle.
- Learn critical concepts around data acquisition techniques.
- Understanding different data storage options like files, SQL and NoSQL.
- Learn application scalability in terms of deployment.
- Python programming basics including data structures, logic, working with files, invoking APIs, using libraries such as Pandas and Numpy, doing ETL.
- SQL query language, SELECT, INSERT, UPDATE, DELETE statements, database functions, stored procs, working with multiple tables, JOINs, & transactions.
- Schedule, automate, and monitor data pipelines using Apache Airflow. Run data quality checks, track data lineage, and work with data pipelines in production.
- Orchestrate Data with Airflow.
Become a Certified Data Science Professional
with this programme
Become a certified DevOps professional with this fast track programme
Course Curriculum
Here is what's included in this DevOps Course
- Python from Basics to Intermediate
- SQL from Basics to Advanced
- Python Data Structures
- Working with Data in Python
- Basics of Python + Code Best Practices using Python
- Tableau would be used for visualization
- Relational Database Concepts
- Using Relational Databases
- MySQL
- Getting Started with SQL
- Introduction to Relational Databases and Tables
- Intermediate SQL
- Accessing Databases using Python
- Data Science to Data Engineering Foundation
Course Curriculum
Here is what's included in this DevOps Course
- Python Basics
- Python Data Structures
- Working with Data in Python
- Basics of Python + Code Best Practices using Python
- Git (basic implementation & advance concepts)
- Fetching & Posting data using apis (postman/python)
- Hands-on: Python Request Library
- Authentication for APIs
- Oauth 2.0 flow
- Hands-on: Fetching data from Youtube Data API
- Hands-on: Google Sheets API
- Any public API listed here
- Why scrape the web?
- How Does Web Scraping Work?
- Techniques to scrape data from web
- Hands-on: Web Scraping
- Custom web Scraper (Bs4, requests)
- Introduction to Scrapy
Shell Scripting
- Linux & Command Line Basics
- Linux shell: Setup
- What is the shell?
- Navigating, Creation and Migration commands
- Differentiate and Pipes
Job Scheduling
- Crontab in Linux
- Scheduling Cron jobs: Linux
- Scheduling Tasks with At Utility: Windows
- Relational Database Concepts
- Using Relational Databases
- MySQL and PostgreSQL
- Introduction to Cloud databases
- Cloud Storage types and uses cases
- Getting data from cloud database
- Getting Started with SQL
- Introduction to Relational Databases and Tables
- Intermediate SQL
- Accessing Databases using Python
- Introduction to data pipelines
- ELT Basics
- Comparing ETL & ELT
- Data Extraction Techniques
- Introduction to Data Transformation Techniques
- ETL using Shell Scripting
- Batch Versus Streaming Data Pipeline Use Cases
- Use Case: Building an Advanced ETL pipeline
- Apache Airflow Overview
- Advantage of Using Data Pipelines as DAGs
- Apache Airflow UI
- Build DAG Using Airflow
- Airflow Monitoring and Logging
- Hands-on: Writing DAG
- Distributed Event Streaming Platform Components
- Apache Kafka Overview
- Building Event Streaming Pipelines using Kafka
- Kafka Streaming Process
- Data Warehouses, Data Marts, and Data Lakes
- Designing, Modeling and Implementing Data Warehouses
- Data Warehouse Analytics
- Introducing NoSQL
- Introducing MongoDB – An Open-Source NoSQL Database
- Introducing Apache Cassandra – An Open-Source NoSQL Database
- Hands-on: Working with NoSQL Databases
- Introduction to Hadoop and Spark
- introduction to MapRedude
- Spark for Data Engineering
- SparkML
Entry Requirements
There are no formal requirements for this certification.
Exams & Certification
Number of questions: 40
Duration: 60 minutes Passing score: 65% (26/40)
Domains:
- Introducing DevOps (5%)
- Benefits of DevOps (10%)
- Culture (15%)
- Automation (15%)
- Lean (10%)
- Measurement (10%)
- Sharing (10%)
- Common DevOps Roles (5%)
Meet Our Trainers
Our experienced Python Programming Language expert are here to help you understand the concepts of python programming and to apply them practically for your successful career
USMAN SHERDIL
Founder & CEO Sherdil IT Academy and Sherdil Cloud
BE Telecommunication & Networks
Course Materials and Support
Online Access to Course Using any Device with online method of delivery while being taught by BCS Accredited Trainer. Track Your Progress with Easy Course Navigation, Quizzes, Videos, Discussions Area.
BCS Certificate
On Successful Completion of the Exam, Receive a Professional Certificate, Accredited by BCS & Aligned to Leading IT Industry Framework - SFIA plus. Globally recognised in over 190 Countries.
BCS Membership
On Successful Completion of Exam, Receive One Free Year BCS Associate Membership (worth £92) Which includes Career Support via Springboard.
Course Materials and Support
Online Access to Course Using any Device with online method of delivery while being taught by BCS Accredited Trainer. Track Your Progress with Easy Course Navigation, Quizzes, Videos, Discussions Area.
BCS Certificate
On Successful Completion of the Exam, Receive a Professional Certificate, Accredited by BCS & Aligned to Leading IT Industry Framework - SFIA plus. Globally recognised in over 190 Countries.
BCS Membership
On Successful Completion of Exam, Receive One Free Year BCS Associate Membership (worth £92) Which includes Career Support via Springboard.
Programming languages and tools
Meet Our Trainers
Our experienced Data Engineering expert are here to help you understand the concepts of Data engineering and to apply them practically for your successful career
More Questions?
Get in touch
More Questions?
Get in touch