Spark机器学习（影印版 英文版）

Preface

Chapter 1： Getting Up and Running with Spark

Installing and setting up Spark locally

Spark clusters

The Spark programming model

SparkContext and SparkConf

The Spark shell

Resilient Distributed Datasets

Creating RDDs

Spark operations

Caching RDDs

Broadcast variables and accumulators

The first step to a Spark program in Scala

The first step to a Spark program in Java

The first step to a Spark program in Python

Getting Spark running on Amazon EC2

Launching an EC2 Spark cluster

Summary

Chapter 2： Designing a Machine Learning System

Introducing MovieStream

Business use cases for a machine learning system

Personalization

Targeted marketing and customer segmentation

Predictive modeling and analytics

Types of machine learning models

The components of a data-driven machine learning system

Data ingestion and storage

Data cleansing and transformation

Model training and testing loop

Model deployment and integration

Model monitoring and feedback

Batch versus real time

An architecture for a machine learning system

Practical exercise

Summary

Chapter 3： Obtaining， Processing， and Preparing Data

with Spark

Accessing publicly available datasets

The MovieLens lOOk dataset

Exploring and visualizing your data

Exploring the user dataset

Exploring the movie dataset

Exploring the rating dataset

Processing and transforming your data

Filling in bad or missing data

Extracting useful features from your data

Numerical features

Categorical features

Derived features

Transforming timestamps into categorical features

Text features

Simple text feature extraction

Normalizing features

Using MLlib for feature normalization

Using packages for feature extraction

Summary

Chapter 4： Building a Recommendation Engine with Spark

Types of recommendation models

Content-based filtering

Collaborative filtering

Matrix factorization

Extracting the right features from your data

Extracting features from the MovieLens 100k dataset

Training the recommendation model

Training a model on the MovieLens 100k dataset

Training a model using implicit feedback data

Using the recommendation model

User recommendations

Generating movie recommendations from the MovieLens 100k dataset

Item recommendations

Generating similar movies for the MovieLens 100k dataset

Evaluating the performance of recommendation models

Mean Squared Error

Mean average precision at K

Using MLlib's built-in evaluation functions

RMSE and MSE

MAP

Summary

Chapter 5： Building a Classification Model with Spark

Types of classification models

Linear models

Logistic regression

Linear support vector machines

The na'fve Bayes model

Decision trees

Extracting the right features from your data

Extracting features from the Kaggle/StumbleUpon

evergreen classification dataset

Training classification models

Training a classification model on the Kaggle/StumbleUpon

evergreen classification dataset

Using classification models

Generating predictions for the Kaggle/StumbleUpon

evergreen classification dataset

Evaluating the performance of classification models

Accuracy and prediction error

Precision and recall

ROC curve and AUC

Improving model performance and tuning parameters

Feature standardization

Additional features

Using the correct form of data

Tuning model parameters

Linear models

Decision trees

The na'fve Bayes model

Cross-validation

Summary

Chapter 6： Buildin a~ssion Model with Spark

Types of regression models

Least squares regression

Decision trees for regression

Extracting the right features from your data

Extracting features from the bike sharing dataset

Creating feature vectors for the linear model

Creating feature vectors for the decision tree

Training and using regression models

Training a regression model on the bike sharing dataset

Evaluating the performance of regression models

Mean Squared Error and Root Mean Squared Error

Mean Absolute Error

Root Mean Squared Log Error

The R-squared coefficient

Computing performance metrics on the bike sharing dataset

Linear model

Decision tree

Improving model performance and tuning parameters

Transforming the target variable

Impact of training on log-transformed targets

Tuning model parameters

Creating training and testing sets to evaluate parameters

The impact of parameter settings for linear models

The impact of parameter settings for the decision tree

Summary

Chapter 7： Building a Clustering Model with Spark

Types of clustering models

K-means clustering

Initialization methods

Variants

Mixture models

Hierarchical clustering

Extracting the right features from your data

Extracting features from the MovieLens dataset

Extracting movie genre labels

Training the recommendation model

Normalization

Training a clustering model

Training a clustering model on the MovieLens dataset

Making predictions using a clustering model

Interpreting cluster predictions on the MovieLens dataset

Interpreting the movie clusters

Evaluating the performance of clustering models

Internal evaluation metrics

External evaluation metrics

Computing performance metrics on the MovieLens dataset

Tuning parameters for clustering models

Selecting K through cross-validation

Summary

Chapter 8： Dimensionality Reduction with Spark

Types of dimensionality reduction

Principal Components Analysis

Singular Value Decomposition

Relationship with matrix factorization

Clustering as dimensionality reduction

Extracting the right features from your data

Extracting features from the LFW dataset

Exploring the face data

Visualizing the face data

Extracting facial images as vectors

Normalization

Training a dimensionality reduction model

Running PCA on the LFW dataset

Visualizing the Eigenfaces

Interpreting the Eigenfaces

Using a dimensionality reduction model

Projecting data using PCA on the LFW dataset

The relationship between PCA and SVD

Evaluating dimensionality reduction models

Evaluating k for SVD on the LFW dataset

Summary

Chapter 9： Advanced Text Processing with Spark

What's so special about text data?

Extracting the right features from your data

Term weighting schemes

Feature hashing

Extracting the TF-IDF features from the 20 Newsgroups dataset

Exploring the 20 Newsgroups data

Applying basic tokenization

Improving our tokenization

Removing stop words

Excluding terms based on frequency

A note about stemming

Training a TF-IDF model

Analyzing the TF-IDF weightings

Using a TF-IDF model

Document similarity with the 20 Newsgroups dataset and

TF-IDF features

Training a text classifier on the 20 Newsgroups dataset

using TF-IDF

Evaluating the impact of text processing

Comparing raw features with processed TF-IDF features on the

20 Newsgroups dataset

Word2Vec models

Word2Vec on the 20 Newsgroups dataset

Summary

Chapter 10： Real-time Machine Learning withSpark Streaming

Online learning

Stream processing

An introduction to Spark Streaming

Input sources

Transformations

Actions

Window operators

Caching and fault tolerance with Spark Streaming

Creating a Spark Streaming application

The producer application

Creating a basic streaming application

Streaming analytics

Stateful streaming

Online learning with Spark Streaming

Streaming regression

A simple streaming regression program

Creating a streaming data producer

Creating a streaming regression model

Streaming K-means

Online model evaluation

Comparing model performance with Spark Streaming

Summary

Index