Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Structure.

• Biological vs. artificial neurons.
• The mathematical model of an ANN.
• Activation functions applied in ANNs.
• Common categories of network architectures.

Mathematical Foundations and Learning Mechanisms.

• Review of vector and matrix algebra.
• Understanding state-space concepts.
• Key principles of optimization.
• Error-correction learning paradigms.
• Memory-based learning approaches.
• Hebbian learning rules.
• Competitive learning strategies.

Single Layer Perceptrons.

• Perceptron structure and training.
• Pattern classifiers: Introduction and Bayes' theorem.
• Utilizing perceptrons as pattern classifiers.
• Perceptron convergence properties.
• Inherent limitations of perceptrons.

Feedforward ANN.

• Architecture of Multi-layer feedforward networks.
• The Back Propagation algorithm.
• Back propagation: Training dynamics and convergence.
• Functional approximation via back propagation.
• Practical considerations and design challenges in back propagation learning.

Radial Basis Function (RBF) Networks.

• Pattern separability and interpolation techniques.
• Theory of Regularization.
• Regularization applied to RBF networks.
• Design and training of RBF networks.
• Approximation capabilities of RBF models.

Competitive Learning and Self-Organizing ANNs.

• Standard clustering procedures.
• Learning Vector Quantization (LVQ).
• Competitive learning algorithms and network structures.
• Self-Organizing Feature Maps (SOFM).
• Characteristics of feature maps.

Fuzzy Neural Networks.

• Neuro-fuzzy systems overview.
• Foundations of fuzzy sets and logic.
• Designing fuzzy systems.
• Designing fuzzy ANNs.

Applications

• Discussion of several Neural Network use cases, highlighting their benefits and potential challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent scenarios
  • Guarantees for finite hypothesis sets – inconsistent scenarios
  • General theoretical contexts
    • Deterministic vs. Stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Model selection strategies
• Rademacher Complexity and VC Dimension
• The Bias-Variance tradeoff
• Regularisation techniques
• Managing Over-fitting
• Model Validation
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self Organisation Maps (SOM)
• Kernel-induced vector spaces
  • Mercer Kernels and Kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

Content will be contextualized against topics covered on Day 1 and Day 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA and Whitening
• Self-Taught Learning
• Deep Networks
• Linear Decoders
• Convolution and Pooling
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demos and Applications