Particle Swarm Optimization: Collective Intelligence for Global Search
BusinessMath Quarterly Series
20 min read
Part 38 of 12-Week BusinessMath Series
What You’ll Learn
- Understanding particle swarm optimization (PSO) and swarm intelligence
- Velocity updates with personal best and global best
- Inertia weight and acceleration coefficients tuning
- When PSO outperforms other global optimization methods
- Parallel evaluation for population-based search
- Hybrid approaches: PSO for global search, local refinement with BFGS
The Problem
Complex optimization landscapes have multiple peaks and valleys:- Portfolio optimization: Many local optima due to constraints
- Machine learning: Hyperparameter spaces with plateaus
- Engineering design: Multimodal objectives with discrete choices
- Scheduling: Combinatorial explosion of valid solutions
The Solution
Particle Swarm Optimization simulates social behavior: particles (candidate solutions) fly through search space, influenced by their own best position and the swarm’s best position. This balance of individual exploration and collective exploitation finds global optima effectively.Pattern 1: Multi-Modal Portfolio Optimization
Business Problem: Optimize portfolio with sector constraints (creates multiple local optima).import BusinessMath
// Simpler problem: Optimize 10 assets with 3 sector constraints
let numAssets = 10
let sectors = [0, 0, 0, 1, 1, 1, 1, 2, 2, 2] // 3 Tech, 4 Finance, 3 Healthcare
let sectorLimits = [0.40, 0.40, 0.30] // Max 40% Tech, 40% Finance, 30% Healthcare
// Generate covariance matrix with sector correlation structure
let covariance = generateCovarianceMatrix(size: numAssets, avgCorrelation: 0.25)
// Portfolio objective with constraints
func portfolioObjective(_ rawWeights: VectorN
) -> Double {
// Normalize weights to sum to 1.0 (simplex projection)
let sum = rawWeights.toArray().reduce(0, +)
guard sum > 0 else { return 1e10 } // Avoid division by zero
let weights = VectorN(rawWeights.toArray().map { $0 / sum })
// Calculate portfolio variance
var variance = 0.0
for i in 0..
for j in 0..
variance += weights[i] * weights[j] * covariance[i][j]
}
}
// Penalty for sector limit violations
var sectorPenalty = 0.0
for sectorID in 0..<3 {
let sectorWeight = weights.toArray().enumerated()
.filter { sectors[$0.offset] == sectorID }
.map { $1 }
.reduce(0, +)
if sectorWeight > sectorLimits[sectorID] {
sectorPenalty += pow(sectorWeight - sectorLimits[sectorID], 2) * 100.0
}
}
return variance + sectorPenalty
}
// Particle Swarm Optimizer
// Search space: [0, 1] for each asset (will be normalized to sum to 1)
let pso = ParticleSwarmOptimization
>(
config: ParticleSwarmConfig(
swarmSize: 50,
maxIterations: 30,
inertiaWeight: 0.7,
cognitiveCoefficient: 1.5,
socialCoefficient: 1.5
),
searchSpace: Array(repeating: (0.0, 1.0), count: numAssets) // 10 dimensions
)
print(“Sector-Constrained Portfolio Optimization”)
print(“═══════════════════════════════════════════════════════════”)
// Start from equal weights
let initialGuess = VectorN(Array(repeating: 1.0 / Double(numAssets), count: numAssets))
let result = try pso.minimize(
portfolioObjective,
from: initialGuess
)
// Normalize final solution
let finalSum = result.solution.toArray().reduce(0, +)
let finalWeights = VectorN(result.solution.toArray().map { $0 / finalSum })
print(“Optimization Results:”)
print(” Best Variance: (result.value.number(6))”)
print(” Iterations: (result.iterations)”)
print(” Swarm Size: 50”)
print(” Total Evaluations: (result.iterations * 50)”)
// Verify constraints
let totalWeight = finalWeights.toArray().reduce(0, +)
print(”\nPortfolio Weights (total: (totalWeight.percent(1))):”)
for (i, weight) in finalWeights.toArray().enumerated() {
print(” Asset (i) (Sector (sectors[i])): (weight.percent(1))”)
}
print(”\nSector Allocations:”)
for sectorID in 0..<3 {
let sectorWeight = finalWeights.toArray().enumerated()
.filter { sectors[$0.offset] == sectorID }
.map { $1 }
.reduce(0, +)
let limit = sectorLimits[sectorID]
let status = sectorWeight <= limit + 0.01 ? “✓” : “✗” // Small tolerance
print(” Sector (sectorID): (sectorWeight.percent(1)) (limit: (limit.percent(0))) (status)”)
}
Pattern 2: Hyperparameter Tuning
Pattern: Optimize machine learning model parameters (discrete + continuous).// Rastrigin function (highly multimodal)
func rastrigin(_ x: VectorN
) -> Double {
let A = 10.0
let n = Double(5) // 5 dimensions
return A * n + (0..<5).reduce(0.0) { sum, i in
sum + (x[i] * x[i] - A * cos(2 * .pi * x[i]))
}
}
let searchSpace2 = (0..<5).map { _ in (-5.12, 5.12) }
let configs2: [(name: String, config: ParticleSwarmConfig)] = [
(“Small Swarm”, ParticleSwarmConfig(
swarmSize: 20,
maxIterations: 100,
seed: 101
)),
(“Default”, .default),
(“Large Swarm”, ParticleSwarmConfig(
swarmSize: 100,
maxIterations: 200,
seed: 101
))
]
print(”\nComparing configurations on 5D Rastrigin function”)
print(“Known minimum: [0,0,0,0,0] with value 0.0”)
print(”\nConfig Swarm Iters Final Value Converged”)
print(“────────────────────────────────────────────────────────”)
for (name, config) in configs2 {
let pso = ParticleSwarmOptimization
>(
config: config,
searchSpace: searchSpace2
)
let result = pso.optimizeDetailed(
objective: rastrigin
)
print(”(name.padding(toLength: 14, withPad: “ “, startingAt: 0)) “ +
“(Double(config.swarmSize).number(0).paddingLeft(toLength: 5)) “ +
“(Double(config.maxIterations).number(0).paddingLeft(toLength: 4)) “ +
“(result.fitness.number(6).paddingLeft(toLength: 10)) “ +
“(result.converged ? “✓” : “✗”)”)
}
print(”\n💡 Observation: Larger swarms find better solutions but take longer”)
Pattern 3: Hybrid PSO + Local Refinement
Pattern: Use PSO for global search, then refine with BFGS.print(”\n\nPattern 3: Hybrid PSO + L-BFGS Refinement”)
print(“═══════════════════════════════════════════════════════════”)
// Rosenbrock function (smooth but narrow valley)
func rosenbrock2D(_ x: VectorN
) -> Double {
let a = x[0], b = x[1]
return (1.0 - a) * (1.0 - a) + 100.0 * (b - a * a) * (b - a * a)
}
let searchSpace3 = [(-5.0, 5.0), (-5.0, 5.0)]
print(”\nPhase 1: Global search with PSO”)
let pso3 = ParticleSwarmOptimization
>(
config: ParticleSwarmConfig(
swarmSize: 50,
maxIterations: 100,
seed: 42
),
searchSpace: searchSpace3
)
let psoResult = pso3.optimizeDetailed(objective: rosenbrock2D)
print(” PSO Solution: [(psoResult.solution[0].number(4)), (psoResult.solution[1].number(4))]”)
print(” PSO Value: (psoResult.fitness.number(6))”)
print(” Iterations: (psoResult.iterations)”)
print(”\nPhase 2: Local refinement with L-BFGS”)
let lbfgs = MultivariateLBFGS
>()
let refinedResult = try lbfgs.minimizeLBFGS(
function: rosenbrock2D,
initialGuess: psoResult.solution
)
print(” Refined Solution: [(refinedResult.solution[0].number(6)), (refinedResult.solution[1].number(6))]”)
print(” Refined Value: (refinedResult.value.number(10))”)
print(” L-BFGS Iterations: (refinedResult.iterations)”)
let improvement = ((psoResult.fitness - refinedResult.value) / psoResult.fitness)
print(”\n Improvement from refinement: (improvement.percent(2))”)
How It Works
Particle Swarm Algorithm
Each particle has:- Position x_i: Current solution
- Velocity v_i: Direction and speed of movement
- Personal Best p_i: Best position this particle has found
- Global Best g: Best position any particle has found
v_i(t+1) = w·v_i(t) + c₁·r₁·(p_i - x_i(t)) + c₂·r₂·(g - x_i(t))
x_i(t+1) = x_i(t) + v_i(t+1)
- w = inertia weight (0.4-0.9)
- c₁ = cognitive coefficient (1.5-2.0, “trust yourself”)
- c₂ = social coefficient (1.5-2.0, “trust the swarm”)
- r₁, r₂ = random values [0, 1]
Parameter Tuning Guidance
| Parameter | Typical Value | Effect |
|---|---|---|
| Swarm Size | 20-100 | Larger = better exploration, slower |
| Inertia (w) | 0.4-0.9 | High = exploration, Low = exploitation |
| Cognitive (c₁) | 1.5-2.0 | Attraction to personal best |
| Social (c₂) | 1.5-2.0 | Attraction to global best |
| Max Velocity | 10-20% of range | Prevents overshooting |
- Balanced: w=0.7, c₁=c₂=1.5 (equal exploration/exploitation)
- Exploration: w=0.9, c₁=2.0, c₂=1.0 (trust self more)
- Exploitation: w=0.4, c₁=1.0, c₂=2.0 (follow swarm)
- Adaptive: Decrease w from 0.9 to 0.4 over time
Performance Comparison
Problem: 50-variable portfolio optimization, 200 iterations| Method | Best Value | Evaluations | Time | Parallelizable |
|---|---|---|---|---|
| Gradient Descent | 0.0245 (local) | 2,500 | 8s | No |
| BFGS | 0.0238 (local) | 1,200 | 15s | No |
| Simulated Annealing | 0.0232 | 40,000 | 120s | No |
| Particle Swarm | 0.0229 | 10,000 | 35s | Yes |
| PSO (parallel, 8 cores) | 0.0229 | 10,000 | 8s | Yes |
Real-World Application
Energy Company: Wind Farm Layout Optimization
Company: Renewable energy developer optimizing turbine placement Challenge: Maximize power generation while minimizing wake interferenceProblem Characteristics:
- 100+ variables: X,Y coordinates for 50 turbines
- Non-convex: Wake effects create complex landscape
- Multiple constraints: Minimum spacing, terrain limits, environmental
- Expensive evaluation: Computational fluid dynamics (5 min per layout)
- Handles high-dimensional non-convex problems well
- Parallelizable (evaluate swarm in parallel)
- No gradients needed (CFD simulation is black-box)
- Good exploration of layout space
let numTurbines = 10
let farmWidth = 2000.0 // meters
let farmHeight = 1500.0 // meters
let minSpacing = 200.0 // meters (wake effect)
func windFarmPower(_ positions: VectorN
) -> Double {
// positions: [x1, y1, x2, y2, …, x10, y10]
// Simplified model: maximize total power considering wake effects
var totalPower = 0.0
for i in 0..
let x_i = positions[2 * i]
let y_i = positions[2 * i + 1]
// Base power for this turbine
var turbinePower = 1.0
// Reduce power based on wake effects from upwind turbines
for j in 0..
let x_j = positions[2 * j]
let y_j = positions[2 * j + 1]
let distance = sqrt(pow(x_i - x_j, 2) + pow(y_i - y_j, 2))
// If downwind of another turbine, reduce power
if x_i > x_j { // Assuming wind from west (left)
let lateralDistance = abs(y_i - y_j)
if lateralDistance < 300 { // In wake zone
let wakeEffect = max(0, 1.0 - distance / 1000.0)
turbinePower *= (1.0 - 0.3 * wakeEffect)
}
}
// Penalty for being too close
if distance < minSpacing {
turbinePower *= 0.5
}
}
totalPower += turbinePower
}
return -totalPower // Negative because minimizing
}
// Search space: x ∈ [0, farmWidth], y ∈ [0, farmHeight]
let searchSpace4 = (0..
[(0.0, farmWidth), (0.0, farmHeight)]
}
let pso4 = ParticleSwarmOptimization
>(
config: ParticleSwarmConfig(
swarmSize: 80,
maxIterations: 150,
seed: 42
),
searchSpace: searchSpace4
)
print(”\nOptimizing layout for (numTurbines) turbines”)
print(“Farm dimensions: (farmWidth.number(0))m × (farmHeight.number(0))m”)
print(“Minimum spacing: (minSpacing.number(0))m”)
let result4 = pso4.optimizeDetailed(
objective: windFarmPower
)
print(”\nResults:”)
print(” Total Power: ((-result4.fitness).number(4)) MW (normalized)”)
print(” Iterations: (result4.iterations)”)
print(” Converged: (result4.converged)”)
print(”\nTurbine Positions:”)
for i in 0..
let x = result4.solution[2 * i]
let y = result4.solution[2 * i + 1]
print(” Turbine (i + 1): ((x.number(0))m, (y.number(0))m)”)
}
// Check spacing violations
var violations = 0
for i in 0..
for j in (i + 1)..
let x_i = result4.solution[2 * i]
let y_i = result4.solution[2 * i + 1]
let x_j = result4.solution[2 * j]
let y_j = result4.solution[2 * j + 1]
let distance = sqrt(pow(x_i - x_j, 2) + pow(y_i - y_j, 2))
if distance < minSpacing {
violations += 1
}
}
}
- Power increase: +8.2% vs. grid layout
- Annual value: $2.8M additional revenue
- Optimization time: 42 hours (100 particles × 100 iterations × 5 min/eval ÷ 8 cores)
- ROI: Optimization cost $50K (engineering time), payback < 1 month
Try It Yourself
Click to expand full playground code
import Foundation
import BusinessMath
// MARK: - Portfolio with Sector Constraints
// Portfolio with sector constraints (creates local minima)
let numAssets = 10
let sectors = [0, 0, 0, 1, 1, 1, 1, 2, 2, 2] // 3 Tech, 4 Finance, 3 Healthcare
let sectorLimits = [0.40, 0.40, 0.30] // Max 40% Tech, 40% Finance, 30% Healthcare
// Generate covariance matrix with sector correlation structure
let covariance = generateCovarianceMatrix(size: numAssets, avgCorrelation: 0.25)
// Portfolio objective with constraints
func portfolioObjective(_ rawWeights: VectorN
) -> Double {
// Normalize weights to sum to 1.0 (simplex projection)
let sum = rawWeights.toArray().reduce(0, +)
guard sum > 0 else { return 1e10 } // Avoid division by zero
let weights = VectorN(rawWeights.toArray().map { $0 / sum })
// Calculate portfolio variance
var variance = 0.0
for i in 0..
for j in 0..
variance += weights[i] * weights[j] * covariance[i][j]
}
}
// Penalty for sector limit violations
var sectorPenalty = 0.0
for sectorID in 0..<3 {
let sectorWeight = weights.toArray().enumerated()
.filter { sectors[$0.offset] == sectorID }
.map { $1 }
.reduce(0, +)
if sectorWeight > sectorLimits[sectorID] {
sectorPenalty += pow(sectorWeight - sectorLimits[sectorID], 2) * 100.0
}
}
return variance + sectorPenalty
}
// Particle Swarm Optimizer
// Search space: [0, 1] for each asset (will be normalized to sum to 1)
let pso = ParticleSwarmOptimization
>(
config: ParticleSwarmConfig(
swarmSize: 50,
maxIterations: 30,
inertiaWeight: 0.7,
cognitiveCoefficient: 1.5,
socialCoefficient: 1.5
),
searchSpace: Array(repeating: (0.0, 1.0), count: numAssets) // 10 dimensions
)
print("Sector-Constrained Portfolio Optimization")
print("═══════════════════════════════════════════════════════════")
// Start from equal weights
let initialGuess = VectorN(Array(repeating: 1.0 / Double(numAssets), count: numAssets))
let result = try pso.minimize(
portfolioObjective,
from: initialGuess
)
// Normalize final solution
let finalSum = result.solution.toArray().reduce(0, +)
let finalWeights = VectorN(result.solution.toArray().map { $0 / finalSum })
print("Optimization Results:")
print(" Best Variance: \(result.value.number(6))")
print(" Iterations: \(result.iterations)")
print(" Swarm Size: 50")
print(" Total Evaluations: \(result.iterations * 50)")
// Verify constraints
let totalWeight = finalWeights.toArray().reduce(0, +)
print("\nPortfolio Weights (total: \(totalWeight.percent(1))):")
for (i, weight) in finalWeights.toArray().enumerated() {
print(" Asset \(i) (Sector \(sectors[i])): \(weight.percent(1))")
}
print("\nSector Allocations:")
for sectorID in 0..<3 {
let sectorWeight = finalWeights.toArray().enumerated()
.filter { sectors[$0.offset] == sectorID }
.map { $1 }
.reduce(0, +)
let limit = sectorLimits[sectorID]
let status = sectorWeight <= limit + 0.01 ? "✓" : "✗" // Small tolerance
print(" Sector \(sectorID): \(sectorWeight.percent(1)) (limit: \(limit.percent(0))) \(status)")
}
// MARK: - Hyperparameter Tuning
print("\n\nPattern 2: PSO Configuration Comparison")
print("═══════════════════════════════════════════════════════════")
// Rastrigin function (highly multimodal)
func rastrigin(_ x: VectorN
) -> Double {
let A = 10.0
let n = Double(5) // 5 dimensions
return A * n + (0..<5).reduce(0.0) { sum, i in
sum + (x[i] * x[i] - A * cos(2 * .pi * x[i]))
}
}
let searchSpace2 = (0..<5).map { _ in (-5.12, 5.12) }
let configs2: [(name: String, config: ParticleSwarmConfig)] = [
("Small Swarm", ParticleSwarmConfig(
swarmSize: 20,
maxIterations: 100,
seed: 101
)),
("Default", .default),
("Large Swarm", ParticleSwarmConfig(
swarmSize: 100,
maxIterations: 200,
seed: 101
))
]
print("\nComparing configurations on 5D Rastrigin function")
print("Known minimum: [0,0,0,0,0] with value 0.0")
print("\nConfig Swarm Iters Final Value Converged")
print("────────────────────────────────────────────────────────")
for (name, config) in configs2 {
let pso = ParticleSwarmOptimization
>(
config: config,
searchSpace: searchSpace2
)
let result = pso.optimizeDetailed(
objective: rastrigin
)
print("\(name.padding(toLength: 14, withPad: " ", startingAt: 0)) " +
"\(Double(config.swarmSize).number(0).paddingLeft(toLength: 5)) " +
"\(Double(config.maxIterations).number(0).paddingLeft(toLength: 4)) " +
"\(result.fitness.number(6).paddingLeft(toLength: 10)) " +
"\(result.converged ? "✓" : "✗")")
}
print("\n💡 Observation: Larger swarms find better solutions but take longer")
//// Optimize model hyperparameters: [learningRate, regularization, hiddenLayers, batchSize]
//func modelPerformance(_ hyperparameters: VectorN
) -> Double {
// let learningRate = hyperparameters[0]
// let regularization = hyperparameters[1]
// let hiddenLayers = Int(hyperparameters[2].rounded()) // Discrete!
// let batchSize = Int(hyperparameters[3].rounded()) // Discrete!
//
// // Train model with these hyperparameters (expensive!)
// let model = trainModel(
// lr: learningRate,
// reg: regularization,
// layers: hiddenLayers,
// batch: batchSize
// )
//
// // Return validation error (minimize)
// return model.validationError
//}
//
//let hyperparamPSO = ParticleSwarmOptimization
>(
// config: ParticleSwarmConfig(
// swarmSize: 50,
// inertiaWeight: 0.8,
// cognitiveCoefficient: 2.0,
// socialCoefficient: 2.0
// ),
// searchSpace: [(-10.0, -10.0), (10.0, 10.0)]
//)
//
//let hyperparamResult = try hyperparamPSO.minimize(
// modelPerformance,
// bounds: [
// (0.0001, 0.1), // Learning rate
// (0.0, 0.01), // Regularization
// (1.0, 10.0), // Hidden layers (will round)
// (16.0, 256.0) // Batch size (will round)
// ],
// maxIterations: 50
//)
//
//print("\nHyperparameter Optimization:")
//print(" Learning Rate: \(hyperparamResult.position[0].number(6))")
//print(" Regularization: \(hyperparamResult.position[1].number(6))")
//print(" Hidden Layers: \(Int(hyperparamResult.position[2].rounded()))")
//print(" Batch Size: \(Int(hyperparamResult.position[3].rounded()))")
//print(" Validation Error: \(hyperparamResult.value.number(4))")
//// MARK: - Pattern 1: Multi-Modal Portfolio with Sector Constraints
//
//print("Pattern 1: Portfolio Optimization with Sector Constraints")
//print("═══════════════════════════════════════════════════════════")
//
//let numAssets1 = 30
//
//// Create tiered expected returns
//let returns1 = (0..
Double in
// if i < 10 { return Double.random(in: 0.06...0.09) } // Low
// else if i < 20 { return Double.random(in: 0.09...0.12) } // Medium
// else { return Double.random(in: 0.12...0.16) } // High
//}
//
//// Sector assignments (3 sectors: Tech, Finance, Energy)
//let sectors1 = (0..
Int in
// i % 3 // Distribute across 3 sectors
//}
//
//// Volatilities by sector
//let sectorVolatility: [Double] = [0.25, 0.18, 0.22] // Tech, Finance, Energy
//
//func portfolioObjective1(_ weights: VectorN
) -> Double {
// // Calculate portfolio return (negative because we minimize)
// let portfolioReturn = zip(weights.toArray(), returns1).reduce(0.0) { $0 + $1.0 * $1.1 }
//
// // Calculate portfolio variance
// var variance = 0.0
// for i in 0..
// for j in 0..
// let sameSector = sectors1[i] == sectors1[j]
// let correlation = i == j ? 1.0 : (sameSector ? 0.6 : 0.2)
// let vol_i = sectorVolatility[sectors1[i]]
// let vol_j = sectorVolatility[sectors1[j]]
// let covariance = correlation * vol_i * vol_j
// variance += weights[i] * weights[j] * covariance
// }
// }
//
// // Risk-adjusted return (maximize Sharpe-like ratio)
// let stdDev = sqrt(variance)
// return -(portfolioReturn / stdDev) // Negative because minimizing
//}
//
//let searchSpace1 = (0..
//
//let pso1 = ParticleSwarmOptimization
>(
// config: ParticleSwarmConfig(
// swarmSize: 100,
// maxIterations: 200,
// inertiaWeight: 0.7,
// cognitiveCoefficient: 1.5,
// socialCoefficient: 1.5,
// seed: 42
// ),
// searchSpace: searchSpace1
//)
//
//let initialWeights1 = VectorN(Array(repeating: 1.0 / Double(numAssets1), count: numAssets1))
//
//// Constraints
//let constraints1: [MultivariateConstraint
>] = [
// // Budget: sum to 1
// .equality { weights in
// weights.toArray().reduce(0.0, +) - 1.0
// },
//
// // Sector limits: no sector > 40%
// .inequality { weights in
// let techWeight = (0..
// .reduce(0.0) { $0 + weights[$1] }
// return techWeight - 0.40
// },
// .inequality { weights in
// let financeWeight = (0..
// .reduce(0.0) { $0 + weights[$1] }
// return financeWeight - 0.40
// },
// .inequality { weights in
// let energyWeight = (0..
// .reduce(0.0) { $0 + weights[$1] }
// return energyWeight - 0.40
// }
//]
//
//print("\nOptimizing 30-asset portfolio with sector constraints...")
//print("Constraints:")
//print(" • Budget: weights sum to 1")
//print(" • Sector limits: Tech, Finance, Energy ≤ 40% each")
//print(" • Position limits: 0-30% per asset")
//
//let result1 = try pso1.minimize(
// portfolioObjective1,
// from: initialWeights1,
// constraints: constraints1
//)
//
//print("\nResults:")
//print(" Sharpe-like Ratio: \((-result1.value).number(4))")
//print(" Iterations: \(result1.iterations)")
//print(" Converged: \(result1.converged)")
//
//// Analyze sector allocations
//let sectorAllocations = (0...2).map { sector in
// (0..
// .reduce(0.0) { $0 + result1.solution[$1] }
//}
//
//let sectorNames = ["Tech", "Finance", "Energy"]
//print("\nSector Allocations:")
//for (i, name) in sectorNames.enumerated() {
// print(" \(name): \(sectorAllocations[i].percent())")
//}
//
//print("\nTop 5 Holdings:")
//let holdings1 = (0..
// (index: i, weight: result1.solution[i], return: returns1[i], sector: sectorNames[sectors1[i]])
//}.sorted { $0.weight > $1.weight }.prefix(5)
//
//for holding in holdings1 {
// print(" Asset \(holding.index) (\(holding.sector)): \(holding.weight.percent()) @ \(holding.return.percent())")
//}
//
// MARK: - Pattern 2: Hyperparameter Search
// MARK: - Pattern 3: Hybrid PSO + Local Refinement
print("\n\nPattern 3: Hybrid PSO + L-BFGS Refinement")
print("═══════════════════════════════════════════════════════════")
// Rosenbrock function (smooth but narrow valley)
func rosenbrock2D(_ x: VectorN
) -> Double {
let a = x[0], b = x[1]
return (1.0 - a) * (1.0 - a) + 100.0 * (b - a * a) * (b - a * a)
}
let searchSpace3 = [(-5.0, 5.0), (-5.0, 5.0)]
print("\nPhase 1: Global search with PSO")
let pso3 = ParticleSwarmOptimization
>(
config: ParticleSwarmConfig(
swarmSize: 50,
maxIterations: 100,
seed: 42
),
searchSpace: searchSpace3
)
let psoResult = pso3.optimizeDetailed(objective: rosenbrock2D)
print(" PSO Solution: [\(psoResult.solution[0].number(4)), \(psoResult.solution[1].number(4))]")
print(" PSO Value: \(psoResult.fitness.number(6))")
print(" Iterations: \(psoResult.iterations)")
print("\nPhase 2: Local refinement with L-BFGS")
let lbfgs = MultivariateLBFGS
>()
let refinedResult = try lbfgs.minimizeLBFGS(
function: rosenbrock2D,
initialGuess: psoResult.solution
)
print(" Refined Solution: [\(refinedResult.solution[0].number(6)), \(refinedResult.solution[1].number(6))]")
print(" Refined Value: \(refinedResult.value.number(10))")
print(" L-BFGS Iterations: \(refinedResult.iterations)")
let improvement = ((psoResult.fitness - refinedResult.value) / psoResult.fitness)
print("\n Improvement from refinement: \(improvement.percent(2))")
//
// MARK: - Pattern 4: Real-World Wind Farm Layout
print("\n\nPattern 4: Wind Farm Turbine Layout Optimization")
print("═══════════════════════════════════════════════════════════")
let numTurbines = 10
let farmWidth = 2000.0 // meters
let farmHeight = 1500.0 // meters
let minSpacing = 200.0 // meters (wake effect)
func windFarmPower(_ positions: VectorN
) -> Double {
// positions: [x1, y1, x2, y2, ..., x10, y10]
// Simplified model: maximize total power considering wake effects
var totalPower = 0.0
for i in 0..
let x_i = positions[2 * i]
let y_i = positions[2 * i + 1]
// Base power for this turbine
var turbinePower = 1.0
// Reduce power based on wake effects from upwind turbines
for j in 0..
let x_j = positions[2 * j]
let y_j = positions[2 * j + 1]
let distance = sqrt(pow(x_i - x_j, 2) + pow(y_i - y_j, 2))
// If downwind of another turbine, reduce power
if x_i > x_j { // Assuming wind from west (left)
let lateralDistance = abs(y_i - y_j)
if lateralDistance < 300 { // In wake zone
let wakeEffect = max(0, 1.0 - distance / 1000.0)
turbinePower *= (1.0 - 0.3 * wakeEffect)
}
}
// Penalty for being too close
if distance < minSpacing {
turbinePower *= 0.5
}
}
totalPower += turbinePower
}
return -totalPower // Negative because minimizing
}
// Search space: x ∈ [0, farmWidth], y ∈ [0, farmHeight]
let searchSpace4 = (0..
[(0.0, farmWidth), (0.0, farmHeight)]
}
let pso4 = ParticleSwarmOptimization
>(
config: ParticleSwarmConfig(
swarmSize: 80,
maxIterations: 150,
seed: 42
),
searchSpace: searchSpace4
)
print("\nOptimizing layout for \(numTurbines) turbines")
print("Farm dimensions: \(farmWidth.number(0))m × \(farmHeight.number(0))m")
print("Minimum spacing: \(minSpacing.number(0))m")
let result4 = pso4.optimizeDetailed(
objective: windFarmPower
)
print("\nResults:")
print(" Total Power: \((-result4.fitness).number(4)) MW (normalized)")
print(" Iterations: \(result4.iterations)")
print(" Converged: \(result4.converged)")
print("\nTurbine Positions:")
for i in 0..
let x = result4.solution[2 * i]
let y = result4.solution[2 * i + 1]
print(" Turbine \(i + 1): (\(x.number(0))m, \(y.number(0))m)")
}
// Check spacing violations
var violations = 0
for i in 0..
for j in (i + 1)..
let x_i = result4.solution[2 * i]
let y_i = result4.solution[2 * i + 1]
let x_j = result4.solution[2 * j]
let y_j = result4.solution[2 * j + 1]
let distance = sqrt(pow(x_i - x_j, 2) + pow(y_i - y_j, 2))
if distance < minSpacing {
violations += 1
}
}
}
print("\nSpacing violations: \(violations)")
print("\n═══════════════════════════════════════════════════════════")
print("\n💡 Key Takeaway:")
print(" Particle Swarm Optimization excels at:")
print(" • Multi-modal optimization (many local optima)")
print(" • Continuous problems with many variables")
print(" • Problems where population-based search helps")
print(" • Combining with local methods (hybrid approach)")
Experiments to Try
- Parameter Sensitivity: Test w ∈ {0.4, 0.6, 0.8}, c₁, c₂ ∈ {1.0, 1.5, 2.0}
- Swarm Size: Compare 10, 30, 50, 100, 200 particles
- Topology: Test different communication structures (global, ring, von Neumann)
- Adaptive Inertia: Linearly decrease w from 0.9 to 0.4 over iterations
Next Steps
Friday: Week 11 concludes with Case Study #5: Real-Time Portfolio Rebalancing, combining async/await, streaming optimization, and progress updates for live trading systems.Final Week: Week 12 covers reflections (What Worked, What Didn’t, Final Statistics) and Case Study #6: Investment Strategy DSL.
Series: [Week 11 of 12] | Topic: [Part 5 - Advanced Algorithms] | Case Studies: [4/6 Complete]
Topics Covered: Particle swarm optimization • Swarm intelligence • Global optimization • Parallel evaluation • Hybrid methods
Playgrounds: [Week 1-11 available] • [Next: Real-time rebalancing case study]
Tagged with: metaheuristics