Apache Spark and Scala

Course Features

Course Details

Spark And Hadoop Distributed File System
What is Spark
Comparison with Hadoop
Components of Spark
Spark Components, Common Spark Algorithms-Iterative Algorithms, Graph Analysis, Machine Learning
Apache Spark- Introduction, Consistency, Availability, Partition
Unified Stack Spark
Spark Components
Comparison with Hadoop – Scalding example, mahout, storm, graph
Running Spark On A Cluster, Writing Spark Applications Using Python, Java, Scala
Explain python example
Show installing a spark
Explain driver program
Explaining spark context with example
Define weakly typed variable
Combine scala and java seamlessly
Explain concurrency and distribution
Explain what is trait
Explain higher order function with example
Define OFI scheduler
Advantages of Spark
Example of Lamda using spark
Explain Mapreduce with example
RDD And Its Operation
Difference between RISC and CISC
Define Apache Mesos
Cartesian product between two RDD
Define count
Define Filter
Define Fold
Define API Operations
Define Factors
Spark, Hadoop, And The Enterprise Data Centre, Common Spark Algorithms
How hadoop cluster is different from spark
Define writing data
Explain sequence file and its usefulness
Define protocol buffers
Define text file, CSV, Object Files and File System
Define sparse metrics
Explain RDD and Compression
Explain data stores and its usefulness
Spark Streaming
Define Elastic Search
Explain Streaming and its usefulness
Apache bookeeper
Define Dstream
Define mapreduce word count
Explain Paraquet
Scala ORM
Define Mlib
Explain multi graphix and its usefulness
Define property graph
Spark Persistence In Spark
Persistence
Motivation
Example
Transformation
Scala and Python
Examples – K-means
Latent Dirichlet Allocation (LDA)
Broadcast And Accumulator
Motivation
Broadcast Variables
Example: Join
Alternative if one table is small
Better version with broadcast
How to create a Broadcast
Accumulators motivation
Accumulator Rules
Custom accumulators
Another common use
Creating an accumulator using spark context object
Spark SQL And RDD
Spark SQL main capabilities
Spark SQL usage diagram
Spark SQL
Important topics in Spark SQL- Data frames
Twitter language analysis
Introduction Of Scala
Scala Overview
Pattern Matching
Advantages of Scala
REPL (Read Evaluate print loop)
Language Features
Type Interface
Higher order function
Option
Pattern Matching
Collection
Currying
Traits
Application Space
Executing The Scala Code
Uses of scala interpreter
Example of static object timer in scala
Testing of String equality in scala
Implicit classes in scala with examples
Recursion in scala
Currying in scala with examples
Classes in scala
Classes Concept In Scala
Constructor
Constructor overloading
Prop erties
Abstract classes
Type hierarchy in Scala
Object equality
Val and var methods
Case Classes And Pattern Matching
Sealed traits
Case classes
Constant pattern in case classes
Wild card pattern
Variable pattern
Constructor pattern
Tuple pattern
Concepts Of Traits With Example
Java equivalents
Advantages of traits
Avoiding boilerplate code
Linearization of traits
Modelling a real world example
Scala Java Interoperability
How traits are implemented in scala and java
How extending multiple traits is handled
Scala Collections
Classification of scala collections
Iterable
Iterator and iterable
List sequence example in scala
Mutable Collections Vs. Immutable Collections
Array in scala
List in scala
Difference between list and list buffer
Array buffer
Queue in scala
Dequeue in scala
Mutable queue in scala
Stacks in scala
Sets and maps in scala
Tuples
Use Case Bobsrockets Package
Different import types
Selective imports
Testing-Assertions
Scala test case- scala test fun. Suite
Junit test in scala
Interface for Junit via Junit 3 suite in scala test
SBT
Directory structure for packaging scala application
 
This course does not have any sections.

More Courses by this Instructor