Particle Swarm Optimization: Collective Intelligence for Global Search
BusinessMath Quarterly Series
17 min read
Part 38 of 12-Week BusinessMath Series
What You’ll Learn
- Understanding particle swarm optimization (PSO) and swarm intelligence
- Velocity updates with personal best and global best
- Inertia weight and acceleration coefficients tuning
- When PSO outperforms other global optimization methods
- Parallel evaluation for population-based search
- Hybrid approaches: PSO for global search, local refinement with BFGS
The Problem
Complex optimization landscapes have multiple peaks and valleys:
- Portfolio optimization: Many local optima due to constraints
- Machine learning: Hyperparameter spaces with plateaus
- Engineering design: Multimodal objectives with discrete choices
- Scheduling: Combinatorial explosion of valid solutions
Need global search that explores broadly while converging to optimum.
The Solution
Particle Swarm Optimization simulates social behavior: particles (candidate solutions) fly through search space, influenced by their own best position and the swarm’s best position. This balance of individual exploration and collective exploitation finds global optima effectively.
Pattern 1: Multi-Modal Portfolio Optimization
Business Problem: Optimize portfolio with sector constraints (creates multiple local optima).
import BusinessMath
// Simpler problem: Optimize 10 assets with 3 sector constraints
let numAssets = 10
let sectors = [0, 0, 0, 1, 1, 1, 1, 2, 2, 2] // 3 Tech, 4 Finance, 3 Healthcare
let sectorLimits = [0.40, 0.40, 0.30] // Max 40% Tech, 40% Finance, 30% Healthcare
// Generate covariance matrix with sector correlation structure
let covariance = generateCovarianceMatrix(size: numAssets, avgCorrelation: 0.25)
// Portfolio objective with constraints
func portfolioObjective(_ rawWeights: VectorN
) -> Double { // Normalize weights to sum to 1.0 (simplex projection) let sum = rawWeights.toArray().reduce(0, +) guard sum > 0 else { return 1e10 } // Avoid division by zero let weights = VectorN(rawWeights.toArray().map { $0 / sum }) // Calculate portfolio variance var variance = 0.0 for i in 0..
sectorLimits[sectorID] { sectorPenalty += pow(sectorWeight - sectorLimits[sectorID], 2) * 100.0 } } return variance + sectorPenalty } // Particle Swarm Optimizer // Search space: [0, 1] for each asset (will be normalized to sum to 1) let pso = ParticleSwarmOptimization
>( config: ParticleSwarmConfig( swarmSize: 50, maxIterations: 30, inertiaWeight: 0.7, cognitiveCoefficient: 1.5, socialCoefficient: 1.5 ), searchSpace: Array(repeating: (0.0, 1.0), count: numAssets) // 10 dimensions ) print("Sector-Constrained Portfolio Optimization") print("═══════════════════════════════════════════════════════════") // Start from equal weights let initialGuess = VectorN(Array(repeating: 1.0 / Double(numAssets), count: numAssets)) let result = try pso.minimize( portfolioObjective, from: initialGuess ) // Normalize final solution let finalSum = result.solution.toArray().reduce(0, +) let finalWeights = VectorN(result.solution.toArray().map { $0 / finalSum }) print("Optimization Results:") print(" Best Variance: \(result.value.number(6))") print(" Iterations: \(result.iterations)") print(" Swarm Size: 50") print(" Total Evaluations: \(result.iterations * 50)") // Verify constraints let totalWeight = finalWeights.toArray().reduce(0, +) print("\nPortfolio Weights (total: \(totalWeight.percent(1))):") for (i, weight) in finalWeights.toArray().enumerated() { print(" Asset \(i) (Sector \(sectors[i])): \(weight.percent(1))") } print("\nSector Allocations:") for sectorID in 0..<3 { let sectorWeight = finalWeights.toArray().enumerated() .filter { sectors[$0.offset] == sectorID } .map { $1 } .reduce(0, +) let limit = sectorLimits[sectorID] let status = sectorWeight <= limit + 0.01 ? "✓" : "✗" // Small tolerance print(" Sector \(sectorID): \(sectorWeight.percent(1)) (limit: \(limit.percent(0))) \(status)") }
Pattern 2: Hyperparameter Tuning
Pattern: Optimize machine learning model parameters (discrete + continuous).
// Rastrigin function (highly multimodal)
func rastrigin(_ x: VectorN
) -> Double { let A = 10.0 let n = Double(5) // 5 dimensions return A * n + (0..<5).reduce(0.0) { sum, i in sum + (x[i] * x[i] - A * cos(2 * .pi * x[i])) } } let searchSpace2 = (0..<5).map { _ in (-5.12, 5.12) } let configs2: [(name: String, config: ParticleSwarmConfig)] = [ ("Small Swarm", ParticleSwarmConfig( swarmSize: 20, maxIterations: 100, seed: 101 )), ("Default", .default), ("Large Swarm", ParticleSwarmConfig( swarmSize: 100, maxIterations: 200, seed: 101 )) ] print("\nComparing configurations on 5D Rastrigin function") print("Known minimum: [0,0,0,0,0] with value 0.0") print("\nConfig Swarm Iters Final Value Converged") print("────────────────────────────────────────────────────────") for (name, config) in configs2 { let pso = ParticleSwarmOptimization
>( config: config, searchSpace: searchSpace2 ) let result = pso.optimizeDetailed( objective: rastrigin ) print("\(name.padding(toLength: 14, withPad: " ", startingAt: 0)) " + "\(Double(config.swarmSize).number(0).paddingLeft(toLength: 5)) " + "\(Double(config.maxIterations).number(0).paddingLeft(toLength: 4)) " + "\(result.fitness.number(6).paddingLeft(toLength: 10)) " + "\(result.converged ? "✓" : "✗")") } print("\n💡 Observation: Larger swarms find better solutions but take longer")
Pattern 3: Hybrid PSO + Local Refinement
Pattern: Use PSO for global search, then refine with BFGS.
print("\n\nPattern 3: Hybrid PSO + L-BFGS Refinement")
print("═══════════════════════════════════════════════════════════")
// Rosenbrock function (smooth but narrow valley)
func rosenbrock2D(_ x: VectorN
) -> Double { let a = x[0], b = x[1] return (1.0 - a) * (1.0 - a) + 100.0 * (b - a * a) * (b - a * a) } let searchSpace3 = [(-5.0, 5.0), (-5.0, 5.0)] print("\nPhase 1: Global search with PSO") let pso3 = ParticleSwarmOptimization
>( config: ParticleSwarmConfig( swarmSize: 50, maxIterations: 100, seed: 42 ), searchSpace: searchSpace3 ) let psoResult = pso3.optimizeDetailed(objective: rosenbrock2D) print(" PSO Solution: [\(psoResult.solution[0].number(4)), \(psoResult.solution[1].number(4))]") print(" PSO Value: \(psoResult.fitness.number(6))") print(" Iterations: \(psoResult.iterations)") print("\nPhase 2: Local refinement with L-BFGS") let lbfgs = MultivariateLBFGS
>() let refinedResult = try lbfgs.minimizeLBFGS( function: rosenbrock2D, initialGuess: psoResult.solution ) print(" Refined Solution: [\(refinedResult.solution[0].number(6)), \(refinedResult.solution[1].number(6))]") print(" Refined Value: \(refinedResult.value.number(10))") print(" L-BFGS Iterations: \(refinedResult.iterations)") let improvement = ((psoResult.fitness - refinedResult.value) / psoResult.fitness) print("\n Improvement from refinement: \(improvement.percent(2))")
How It Works
Particle Swarm Algorithm
Each particle has:
- Position x_i: Current solution
- Velocity v_i: Direction and speed of movement
- Personal Best p_i: Best position this particle has found
- Global Best g: Best position any particle has found
Update Equations:
v_i(t+1) = w·v_i(t) + c₁·r₁·(p_i - x_i(t)) + c₂·r₂·(g - x_i(t))
x_i(t+1) = x_i(t) + v_i(t+1)
Where:
- w = inertia weight (0.4-0.9)
- c₁ = cognitive coefficient (1.5-2.0, “trust yourself”)
- c₂ = social coefficient (1.5-2.0, “trust the swarm”)
- r₁, r₂ = random values [0, 1]
Parameter Tuning Guidance
| Parameter | Typical Value | Effect |
|---|---|---|
| Swarm Size | 20-100 | Larger = better exploration, slower |
| Inertia (w) | 0.4-0.9 | High = exploration, Low = exploitation |
| Cognitive (c₁) | 1.5-2.0 | Attraction to personal best |
| Social (c₂) | 1.5-2.0 | Attraction to global best |
| Max Velocity | 10-20% of range | Prevents overshooting |
Common Strategies:
- Balanced: w=0.7, c₁=c₂=1.5 (equal exploration/exploitation)
- Exploration: w=0.9, c₁=2.0, c₂=1.0 (trust self more)
- Exploitation: w=0.4, c₁=1.0, c₂=2.0 (follow swarm)
- Adaptive: Decrease w from 0.9 to 0.4 over time
Performance Comparison
Problem: 50-variable portfolio optimization, 200 iterations
| Method | Best Value | Evaluations | Time | Parallelizable |
|---|---|---|---|---|
| Gradient Descent | 0.0245 (local) | 2,500 | 8s | No |
| BFGS | 0.0238 (local) | 1,200 | 15s | No |
| Simulated Annealing | 0.0232 | 40,000 | 120s | No |
| Particle Swarm | 0.0229 | 10,000 | 35s | Yes |
| PSO (parallel, 8 cores) | 0.0229 | 10,000 | 8s | Yes |
PSO wins on: Global optimum quality, parallelizability
Real-World Application
Energy Company: Wind Farm Layout Optimization
Company: Renewable energy developer optimizing turbine placement Challenge: Maximize power generation while minimizing wake interference
Problem Characteristics:
- 100+ variables: X,Y coordinates for 50 turbines
- Non-convex: Wake effects create complex landscape
- Multiple constraints: Minimum spacing, terrain limits, environmental
- Expensive evaluation: Computational fluid dynamics (5 min per layout)
Why Particle Swarm:
- Handles high-dimensional non-convex problems well
- Parallelizable (evaluate swarm in parallel)
- No gradients needed (CFD simulation is black-box)
- Good exploration of layout space
Implementation:
let numTurbines = 10
let farmWidth = 2000.0 // meters
let farmHeight = 1500.0 // meters
let minSpacing = 200.0 // meters (wake effect)
func windFarmPower(_ positions: VectorN
) -> Double { // positions: [x1, y1, x2, y2, ..., x10, y10] // Simplified model: maximize total power considering wake effects var totalPower = 0.0 for i in 0..
x_j { // Assuming wind from west (left) let lateralDistance = abs(y_i - y_j) if lateralDistance < 300 { // In wake zone let wakeEffect = max(0, 1.0 - distance / 1000.0) turbinePower *= (1.0 - 0.3 * wakeEffect) } } // Penalty for being too close if distance < minSpacing { turbinePower *= 0.5 } } totalPower += turbinePower } return -totalPower // Negative because minimizing } // Search space: x ∈ [0, farmWidth], y ∈ [0, farmHeight] let searchSpace4 = (0..
>( config: ParticleSwarmConfig( swarmSize: 80, maxIterations: 150, seed: 42 ), searchSpace: searchSpace4 ) print("\nOptimizing layout for \(numTurbines) turbines") print("Farm dimensions: \(farmWidth.number(0))m × \(farmHeight.number(0))m") print("Minimum spacing: \(minSpacing.number(0))m") let result4 = pso4.optimizeDetailed( objective: windFarmPower ) print("\nResults:") print(" Total Power: \((-result4.fitness).number(4)) MW (normalized)") print(" Iterations: \(result4.iterations)") print(" Converged: \(result4.converged)") print("\nTurbine Positions:") for i in 0..
Results:
Power increase: +8.2% vs. grid layoutAnnual value: $2.8M additional revenueOptimization time: 42 hours (100 particles × 100 iterations × 5 min/eval ÷ 8 cores)ROI: Optimization cost $50K (engineering time), payback < 1 month
Try It Yourself
Click to expand full playground code
import Foundation
import BusinessMath
// MARK: - Portfolio with Sector Constraints
// Portfolio with sector constraints (creates local minima)
let numAssets = 10
let sectors = [0, 0, 0, 1, 1, 1, 1, 2, 2, 2] // 3 Tech, 4 Finance, 3 Healthcare
let sectorLimits = [0.40, 0.40, 0.30] // Max 40% Tech, 40% Finance, 30% Healthcare
// Generate covariance matrix with sector correlation structure
let covariance = generateCovarianceMatrix(size: numAssets, avgCorrelation: 0.25)
// Portfolio objective with constraints
func portfolioObjective(_ rawWeights: VectorN
) -> Double { // Normalize weights to sum to 1.0 (simplex projection) let sum = rawWeights.toArray().reduce(0, +) guard sum > 0 else { return 1e10 } // Avoid division by zero let weights = VectorN(rawWeights.toArray().map { $0 / sum }) // Calculate portfolio variance var variance = 0.0 for i in 0..
sectorLimits[sectorID] { sectorPenalty += pow(sectorWeight - sectorLimits[sectorID], 2) * 100.0 } } return variance + sectorPenalty } // Particle Swarm Optimizer // Search space: [0, 1] for each asset (will be normalized to sum to 1) let pso = ParticleSwarmOptimization
>( config: ParticleSwarmConfig( swarmSize: 50, maxIterations: 30, inertiaWeight: 0.7, cognitiveCoefficient: 1.5, socialCoefficient: 1.5 ), searchSpace: Array(repeating: (0.0, 1.0), count: numAssets) // 10 dimensions ) print("Sector-Constrained Portfolio Optimization") print("═══════════════════════════════════════════════════════════") // Start from equal weights let initialGuess = VectorN(Array(repeating: 1.0 / Double(numAssets), count: numAssets)) let result = try pso.minimize( portfolioObjective, from: initialGuess ) // Normalize final solution let finalSum = result.solution.toArray().reduce(0, +) let finalWeights = VectorN(result.solution.toArray().map { $0 / finalSum }) print("Optimization Results:") print(" Best Variance: \(result.value.number(6))") print(" Iterations: \(result.iterations)") print(" Swarm Size: 50") print(" Total Evaluations: \(result.iterations * 50)") // Verify constraints let totalWeight = finalWeights.toArray().reduce(0, +) print("\nPortfolio Weights (total: \(totalWeight.percent(1))):") for (i, weight) in finalWeights.toArray().enumerated() { print(" Asset \(i) (Sector \(sectors[i])): \(weight.percent(1))") } print("\nSector Allocations:") for sectorID in 0..<3 { let sectorWeight = finalWeights.toArray().enumerated() .filter { sectors[$0.offset] == sectorID } .map { $1 } .reduce(0, +) let limit = sectorLimits[sectorID] let status = sectorWeight <= limit + 0.01 ? "✓" : "✗" // Small tolerance print(" Sector \(sectorID): \(sectorWeight.percent(1)) (limit: \(limit.percent(0))) \(status)") } // MARK: - Hyperparameter Tuning print("\n\nPattern 2: PSO Configuration Comparison") print("═══════════════════════════════════════════════════════════") // Rastrigin function (highly multimodal) func rastrigin(_ x: VectorN
) -> Double { let A = 10.0 let n = Double(5) // 5 dimensions return A * n + (0..<5).reduce(0.0) { sum, i in sum + (x[i] * x[i] - A * cos(2 * .pi * x[i])) } } let searchSpace2 = (0..<5).map { _ in (-5.12, 5.12) } let configs2: [(name: String, config: ParticleSwarmConfig)] = [ ("Small Swarm", ParticleSwarmConfig( swarmSize: 20, maxIterations: 100, seed: 101 )), ("Default", .default), ("Large Swarm", ParticleSwarmConfig( swarmSize: 100, maxIterations: 200, seed: 101 )) ] print("\nComparing configurations on 5D Rastrigin function") print("Known minimum: [0,0,0,0,0] with value 0.0") print("\nConfig Swarm Iters Final Value Converged") print("────────────────────────────────────────────────────────") for (name, config) in configs2 { let pso = ParticleSwarmOptimization
>( config: config, searchSpace: searchSpace2 ) let result = pso.optimizeDetailed( objective: rastrigin ) print("\(name.padding(toLength: 14, withPad: " ", startingAt: 0)) " + "\(Double(config.swarmSize).number(0).paddingLeft(toLength: 5)) " + "\(Double(config.maxIterations).number(0).paddingLeft(toLength: 4)) " + "\(result.fitness.number(6).paddingLeft(toLength: 10)) " + "\(result.converged ? "✓" : "✗")") } print("\n💡 Observation: Larger swarms find better solutions but take longer") //// Optimize model hyperparameters: [learningRate, regularization, hiddenLayers, batchSize] //func modelPerformance(_ hyperparameters: VectorN
) -> Double { // let learningRate = hyperparameters[0] // let regularization = hyperparameters[1] // let hiddenLayers = Int(hyperparameters[2].rounded()) // Discrete! // let batchSize = Int(hyperparameters[3].rounded()) // Discrete! // // // Train model with these hyperparameters (expensive!) // let model = trainModel( // lr: learningRate, // reg: regularization, // layers: hiddenLayers, // batch: batchSize // ) // // // Return validation error (minimize) // return model.validationError //} // //let hyperparamPSO = ParticleSwarmOptimization
>( // config: ParticleSwarmConfig( // swarmSize: 50, // inertiaWeight: 0.8, // cognitiveCoefficient: 2.0, // socialCoefficient: 2.0 // ), // searchSpace: [(-10.0, -10.0), (10.0, 10.0)] //) // //let hyperparamResult = try hyperparamPSO.minimize( // modelPerformance, // bounds: [ // (0.0001, 0.1), // Learning rate // (0.0, 0.01), // Regularization // (1.0, 10.0), // Hidden layers (will round) // (16.0, 256.0) // Batch size (will round) // ], // maxIterations: 50 //) // //print("\nHyperparameter Optimization:") //print(" Learning Rate: \(hyperparamResult.position[0].number(6))") //print(" Regularization: \(hyperparamResult.position[1].number(6))") //print(" Hidden Layers: \(Int(hyperparamResult.position[2].rounded()))") //print(" Batch Size: \(Int(hyperparamResult.position[3].rounded()))") //print(" Validation Error: \(hyperparamResult.value.number(4))") //// MARK: - Pattern 1: Multi-Modal Portfolio with Sector Constraints // //print("Pattern 1: Portfolio Optimization with Sector Constraints") //print("═══════════════════════════════════════════════════════════") // //let numAssets1 = 30 // //// Create tiered expected returns //let returns1 = (0..
Double in // if i < 10 { return Double.random(in: 0.06...0.09) } // Low // else if i < 20 { return Double.random(in: 0.09...0.12) } // Medium // else { return Double.random(in: 0.12...0.16) } // High //} // //// Sector assignments (3 sectors: Tech, Finance, Energy) //let sectors1 = (0..
Int in // i % 3 // Distribute across 3 sectors //} // //// Volatilities by sector //let sectorVolatility: [Double] = [0.25, 0.18, 0.22] // Tech, Finance, Energy // //func portfolioObjective1(_ weights: VectorN
) -> Double { // // Calculate portfolio return (negative because we minimize) // let portfolioReturn = zip(weights.toArray(), returns1).reduce(0.0) { $0 + $1.0 * $1.1 } // // // Calculate portfolio variance // var variance = 0.0 // for i in 0..
>( // config: ParticleSwarmConfig( // swarmSize: 100, // maxIterations: 200, // inertiaWeight: 0.7, // cognitiveCoefficient: 1.5, // socialCoefficient: 1.5, // seed: 42 // ), // searchSpace: searchSpace1 //) // //let initialWeights1 = VectorN(Array(repeating: 1.0 / Double(numAssets1), count: numAssets1)) // //// Constraints //let constraints1: [MultivariateConstraint
>] = [ // // Budget: sum to 1 // .equality { weights in // weights.toArray().reduce(0.0, +) - 1.0 // }, // // // Sector limits: no sector > 40% // .inequality { weights in // let techWeight = (0..
$1.weight }.prefix(5) // //for holding in holdings1 { // print(" Asset \(holding.index) (\(holding.sector)): \(holding.weight.percent()) @ \(holding.return.percent())") //} // // MARK: - Pattern 2: Hyperparameter Search // MARK: - Pattern 3: Hybrid PSO + Local Refinement print("\n\nPattern 3: Hybrid PSO + L-BFGS Refinement") print("═══════════════════════════════════════════════════════════") // Rosenbrock function (smooth but narrow valley) func rosenbrock2D(_ x: VectorN
) -> Double { let a = x[0], b = x[1] return (1.0 - a) * (1.0 - a) + 100.0 * (b - a * a) * (b - a * a) } let searchSpace3 = [(-5.0, 5.0), (-5.0, 5.0)] print("\nPhase 1: Global search with PSO") let pso3 = ParticleSwarmOptimization
>( config: ParticleSwarmConfig( swarmSize: 50, maxIterations: 100, seed: 42 ), searchSpace: searchSpace3 ) let psoResult = pso3.optimizeDetailed(objective: rosenbrock2D) print(" PSO Solution: [\(psoResult.solution[0].number(4)), \(psoResult.solution[1].number(4))]") print(" PSO Value: \(psoResult.fitness.number(6))") print(" Iterations: \(psoResult.iterations)") print("\nPhase 2: Local refinement with L-BFGS") let lbfgs = MultivariateLBFGS
>() let refinedResult = try lbfgs.minimizeLBFGS( function: rosenbrock2D, initialGuess: psoResult.solution ) print(" Refined Solution: [\(refinedResult.solution[0].number(6)), \(refinedResult.solution[1].number(6))]") print(" Refined Value: \(refinedResult.value.number(10))") print(" L-BFGS Iterations: \(refinedResult.iterations)") let improvement = ((psoResult.fitness - refinedResult.value) / psoResult.fitness) print("\n Improvement from refinement: \(improvement.percent(2))") // // MARK: - Pattern 4: Real-World Wind Farm Layout print("\n\nPattern 4: Wind Farm Turbine Layout Optimization") print("═══════════════════════════════════════════════════════════") let numTurbines = 10 let farmWidth = 2000.0 // meters let farmHeight = 1500.0 // meters let minSpacing = 200.0 // meters (wake effect) func windFarmPower(_ positions: VectorN
) -> Double { // positions: [x1, y1, x2, y2, ..., x10, y10] // Simplified model: maximize total power considering wake effects var totalPower = 0.0 for i in 0..
x_j { // Assuming wind from west (left) let lateralDistance = abs(y_i - y_j) if lateralDistance < 300 { // In wake zone let wakeEffect = max(0, 1.0 - distance / 1000.0) turbinePower *= (1.0 - 0.3 * wakeEffect) } } // Penalty for being too close if distance < minSpacing { turbinePower *= 0.5 } } totalPower += turbinePower } return -totalPower // Negative because minimizing } // Search space: x ∈ [0, farmWidth], y ∈ [0, farmHeight] let searchSpace4 = (0..
>( config: ParticleSwarmConfig( swarmSize: 80, maxIterations: 150, seed: 42 ), searchSpace: searchSpace4 ) print("\nOptimizing layout for \(numTurbines) turbines") print("Farm dimensions: \(farmWidth.number(0))m × \(farmHeight.number(0))m") print("Minimum spacing: \(minSpacing.number(0))m") let result4 = pso4.optimizeDetailed( objective: windFarmPower ) print("\nResults:") print(" Total Power: \((-result4.fitness).number(4)) MW (normalized)") print(" Iterations: \(result4.iterations)") print(" Converged: \(result4.converged)") print("\nTurbine Positions:") for i in 0..
→ Full API Reference: BusinessMath Docs – Particle Swarm Optimization Tutorial
Experiments to Try
- Parameter Sensitivity: Test w ∈ {0.4, 0.6, 0.8}, c₁, c₂ ∈ {1.0, 1.5, 2.0}
- Swarm Size: Compare 10, 30, 50, 100, 200 particles
- Topology: Test different communication structures (global, ring, von Neumann)
- Adaptive Inertia: Linearly decrease w from 0.9 to 0.4 over iterations
Next Steps
Friday: Week 11 concludes with Case Study #5: Real-Time Portfolio Rebalancing, combining async/await, streaming optimization, and progress updates for live trading systems.
Final Week: Week 12 covers reflections (What Worked, What Didn’t, Final Statistics) and Case Study #6: Investment Strategy DSL.
Series: [Week 11 of 12] | Topic: [Part 5 - Advanced Algorithms] | Case Studies: [4/6 Complete]
Topics Covered: Particle swarm optimization • Swarm intelligence • Global optimization • Parallel evaluation • Hybrid methods
Playgrounds: [Week 1-11 available] • [Next: Real-time rebalancing case study]
Tagged with: metaheuristics