Introduction to Machine Learning, Learning Paradigms & PAC Learning

Explanation

Machine Learning develops computational algorithms that discover statistical structures in historical data to make automated predictions on unseen data. In Supervised Learning, given training pairs {(x_i, y_i)}, the objective is to approximate the underlying mapping function f: X -> Y. When target values are continuous, the task is Regression; when discrete, it is Classification. Inductive bias represents the set of assumptions an algorithm uses to predict outputs for inputs it has never encountered.

Key points

• Core Learning Paradigms: Supervised Learning (labeled data), Unsupervised Learning (unlabeled data), Reinforcement Learning (reward/penalty signals).
• Batch Learning vs Online Learning: Batch models train on entire static datasets; Online models update model parameters incrementally as data streams arrive.
• Parametric vs Non-Parametric: Parametric models summarize data with a fixed set of parameters (Linear Regression); Non-parametric models grow parameters with dataset size (k-NN).
• Probably Approximately Correct (PAC) Learning: Theoretical framework assessing the sample complexity required to learn an accurate hypothesis with high probability.

Example

Code examples

# Basic classification pipeline using Scikit-Learn
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
print(f"Dataset split: Train={X_train.shape[0]}, Test={X_test.shape[0]}")

Exam questions

• [8 Marks] State Tom Mitchell's formal definition of machine learning with a concrete example. Compare supervised, unsupervised, and reinforcement learning.
• [6 Marks] Explain parametric vs non-parametric algorithms and explain the concept of Inductive Bias.