Apache Spark Application Performance Tuning Course Overview

This three-day hands-on training course delivers the key concepts and expertise developers need to improve the performance of their Apache Spark applications. During the course, participants will learn how to identify common sources of poor performance in Spark applications, techniques for avoiding or solving them, and best practices for Spark application monitoring.

Target Audience

This course is designed for software developers, engineers, and data scientists who have experience developing Spark applications and want to learn how to improve the performance of their code.

Learning Objective

  • Understand Apache Spark's architecture, job execution, and how techniques such as lazy execution and pipelining can improve runtime performance
  • Evaluate the performance characteristics of core data structures such as RDD and DataFrames
  • Select the file formats that will provide the best performance for your application
  • Identify and resolve performance problems caused by data skew
  • Use partitioning, bucketing, and join optimizations to improve SparkSQL performance
  • Understand the performance overhead of Python-based RDDs, DataFrames, and user-defined functions
  • Take advantage of caching for better application performance
  • Understand how the Catalyst and Tungsten optimizers work
  • Understand how Workload XM can help troubleshoot and proactively monitor Spark applications performance
  • Learn about the new features in Spark 3.0 and specifically how the Adaptive Query Execution engine improves performance

This is a Rare Course and it can be take up to 3 weeks to arrange the training.

You will learn:

Module 1: Spark Architecture
  • RDDs
  • DataFrames and Datasets
  • Lazy Evaluation
  • Pipelining
  • Available Formats Overview
  • Impact on Performance
  • The Small Files Problem
  • The Cost of Inference
  • Mitigating Tactics
  • Recognizing Skew
  • Mitigating Tactics
  • Catalyst Overview
  • Tungsten Overview
  • Denormalization
  • Broadcast Joins
  • Map-Side Operations
  • Sort Merge Joins
  • Partitioned Tables
  • Bucketed Tables
  • Impact on Performance
  • Skewed Joins
  • Bucketed Joins
  • Incremental Joins
  • Pyspark Overhead
  • Scalar UDFs
  • Vector UDFs using Apache Arrow
  • Scala UDFs
  • Caching Options
  • Impact on Performance
  • Caching Pitfalls
  • WXM Overview
  • WXM for Spark Developers
  • Adaptive Number of Shuffle Partitions
  • Skew Joins
  • Convert Sort Merge Joins to Broadcast Joins
  • Dynamic Partition Pruning
  • Dynamic Coalesce Shuffle Partitions
Live Online Training (Duration : 24 Hours) Fee On Request
Group Training
15 Aug - 17 Aug 09:00 AM - 05:00 PM CST
(8 Hours/Day)
05 Sep - 07 Sep 09:00 AM - 05:00 PM CST
(8 Hours/Day)
03 Oct - 05 Oct 09:00 AM - 05:00 PM CST
(8 Hours/Day)
We Offer :
  • 1-on-1 Public - Select your own start date. Other students can be merged.
  • 1-on-1 Private - Select your own start date. You will be the only student in the class.
4 Hours
8 Hours
Week Days
Weekend

Start Time : At any time

12 AM
12 PM

1-On-1 Training is Guaranteed to Run (GTR)
Physical Classroom Training with 4 hours /8 hours option is available. Enquire Now!
Ultra-Fast Track

If you can't spare 24 hours. We can offer you an Ultra-Fast Track for 12 hours
Enquire Now!

Course Prerequisites

Spark examples and hands-on exercises are presented in Python and the ability to program in this language is required. Basic familiarity with the Linux command line is assumed. Basic knowledge of SQL is helpful.

Request More Information

Add Name and Email Address of participant (If different from you)

FAQ's


Yes, fee excludes local taxes.
Yes, we do.
The Fee includes:
  • Courseware
Yes, Koenig Solutions is a Cloudera Learning Partner
Schedule for Group Training is decided by Koenig. Schedule for 1-on-1 is decided by you.
In 1-on-1 you can select your own schedule, other students can be merged but you select the schedule. Choose 1-on-1 if published schedule do not meet your requirement. If you also want a private session, opt for 1-on-1 Public.
Yes.
No, it is not included.
In both, you choose the schedule. In public, other participants can join, Private other participants want to join.
Yes, course requiring practical include hands-on labs.
You can buy online from the page by clicking on "Buy Now". You can view alternate payment method on payment options page.
Yes, you can pay from the course page and flexi page.
Yes, the site is secure by utilizing Secure Sockets Layer (SSL) Technology. SSL technology enables the encryption of sensitive information during online transactions. We use the highest assurance SSL/TLS certificate, which ensures that no unauthorized person can get to your sensitive payment data over the web.
We use the best standards in Internet security. Any data retained is not shared with third parties.
You can request a refund if you do not wish to enroll in the course.
To receive an acknowledgment of your online payment, you should have a valid email address. At the point when you enter your name, Visa, and other data, you have the option of entering your email address. Would it be a good idea for you to decide to enter your email address, confirmation of your payment will be emailed to you.
After you submit your payment, you will land on the payment confirmation screen. It contains your payment confirmation message. You will likewise get a confirmation email after your transaction is submitted.
We do accept all major credit cards from Visa, Mastercard, American Express, and Discover.
Credit card transactions normally take 48 hours to settle. Approval is given right away; however, it takes 48 hours for the money to be moved.
Yes, we do accept partial payments, you may use one payment method for part of the transaction and another payment method for other parts of the transaction.
Yes, if we have an office in your city.
Yes, we do offer corporate training More details
Yes, we do.
Yes, we also offer weekend classes.
Yes, Koenig follows a BYOL(Bring Your Own Laptop) policy.
It is recommended but not mandatory. Being acquainted with the basic course material will enable you and the trainer to move at a desired pace during classes. You can access courseware for most vendors.
Yes, this is our official email address which we use if a recipient is not able to receive emails from our @koenig-solutions.com email address.
Buy-Now. Pay-Later option is available using credit card in USA and India only.
You will receive the digital certificate post training completion via learning enhancement tool after registration.
Yes you can.
Yes, we do. For details go to flexi
You can pay through debit/credit card or bank wire transfer.
Dubai, Goa, Delhi, Bangalore.
Yes you can request your customer experience manager for the same.
Yes of course. 100% refund if training not upto your satisfaction.

Prices & Payments

Yes of course.
Yes, We are

Travel and Visa

Yes we do after your registration for course.

Food and Beverages

Yes.

Others

Says our CEO-
“It is an interesting story and dates back half a century. My father started a manufacturing business in India in the 1960's for import substitute electromechanical components such as microswitches. German and Japanese goods were held in high esteem so he named his company Essen Deinki (Essen is a well known industrial town in Germany and Deinki is Japanese for electric company). His products were very good quality and the fact that they sounded German and Japanese also helped. He did quite well. In 1970s he branched out into electronic products and again looked for a German name. This time he chose Koenig, and Koenig Electronics was born. In 1990s after graduating from college I was looking for a name for my company and Koenig Solutions sounded just right. Initially we had marketed under the brand of Digital Equipment Corporation but DEC went out of business and we switched to the Koenig name. Koenig is difficult to pronounce and marketeers said it is not a good choice for a B2C brand. But it has proven lucky for us.” – Says Rohit Aggarwal (Founder and CEO - Koenig Solutions)
All our trainers are fluent in English . Majority of our customers are from outside India and our trainers speak in a neutral accent which is easily understandable by students from all nationalities. Our money back guarantee also stands for accent of the trainer.
Medical services in India are at par with the world and are a fraction of costs in Europe and USA. A number of our students have scheduled cosmetic, dental and ocular procedures during their stay in India. We can provide advice about this, on request.
Yes, if you send 4 participants, we can offer an exclusive training for them which can be started from Any Date™ suitable for you.