Artificial Neural Networks: Biological vs Artificial, McCulloch-Pitts & Perceptron

Explanation

Artificial Neural Networks (ANNs) model information processing through interconnected directed graphs of simple computing nodes. The McCulloch-Pitts model demonstrated that networks of threshold units can perform fundamental logical operations (AND, OR, NOT). Rosenblatt augmented this by introducing adjustable synaptic weights updated through supervised feedback. However, single-layer networks are fundamentally limited to linear hyperplanes; non-linear separations require multi-layered networks equipped with non-linear activation functions.

Key points

• McCulloch-Pitts Neuron (1943): Binary threshold unit taking binary inputs with excitatory and inhibitory connections; first mathematical model of computation in a neuron.
• Rosenblatt's Perceptron (1958): Introduced learnable real-valued continuous weights and automated convergence theorem.
• Perceptron Learning Rule: w_i := w_i + alpha * (y - y_hat) * x_i; guaranteed to converge if data is linearly separable.
• Minsky and Papert (1969): Proved single-layer perceptrons cannot compute the non-linear XOR function, triggering the first AI Winter.

Example

Code examples

# McCulloch-Pitts Neuron for Logical AND gate
import numpy as np

def mp_neuron_and(x1, x2):
    w1, w2, threshold = 1, 1, 2
    net = x1 * w1 + x2 * w2
    return 1 if net >= threshold else 0

print("AND(0,0):", mp_neuron_and(0, 0)) # 0
print("AND(1,1):", mp_neuron_and(1, 1)) # 1

Exam questions

• [8 Marks] Compare biological neurons with artificial neurons. Explain the McCulloch-Pitts neuron model with a logical AND implementation.
• [6 Marks] State and explain the Perceptron Learning Rule. Prove why a single-layer perceptron fails to separate XOR inputs.