Vector Operations: Foundation for Multivariate Optimization

BusinessMath Quarterly Series

15 min read

Part 25 of 12-Week BusinessMath Series

What You’ll Learn

Understanding the VectorSpace protocol and why it matters
Working with Vector2D, Vector3D, and VectorN types
Performing vector operations: norms, dot products, projections
Using generic algorithms that work across all vector types
Creating type-safe multivariate constraints
Building portfolio weights and feature vectors

The Problem

Multivariate optimization requires working with vectors of different dimensions:

Portfolio optimization: N-dimensional weights for N assets
Pricing models: 2D or 3D parameter spaces
Machine learning: High-dimensional feature vectors
Generic algorithms: Code that works for any dimension

Without a unified vector abstraction, you’d write duplicate code for each dimension (optimize2D, optimize3D, optimizeND, etc.).

The Solution

BusinessMath’s VectorSpace protocol provides a generic interface for vector operations. Write optimization algorithms once, they work for all dimensions.

The VectorSpace Protocol

A vector space is a mathematical structure supporting:

Vector addition: v + w
Scalar multiplication: α · v
Zero element: 0
Norms and distances: ‖v‖, ‖v - w‖

Protocol Definition (simplified):

public protocol VectorSpace: AdditiveArithmetic {
    associatedtype Scalar: Real

    // Required operations
    static var zero: Self { get }
    static func + (lhs: Self, rhs: Self) -> Self
    static func * (lhs: Scalar, rhs: Self) -> Self

    // Norm and distance
    var norm: Scalar { get }
    func dot(_ other: Self) -> Scalar

    // Conversion
    static func fromArray(_ array: [Scalar]) -> Self?
    func toArray() -> [Scalar]
}

Why it matters:

// ❌ Before: Duplicate implementations
func optimize2D(_ f: (Vector2D) -> Double, …) -> Vector2D
func optimize3D(_ f: (Vector3D) -> Double, …) -> Vector3D
func optimizeND(_ f: (VectorN) -> Double, …) -> VectorN

// ✅ After: One generic implementation
func optimize
            
              (_ f: (V) -> V.Scalar, …) -> V

One algorithm works for all vector types!

Vector Implementations

BusinessMath provides three vector types optimized for different use cases.

Vector2D: Fixed 2D Vectors

Use Cases:

Two-variable optimization
Coordinate systems (x, y)
Complex numbers (real, imaginary)

Performance: Fastest (compile-time optimization, zero array overhead)

import BusinessMath

// Create a 2D vector
let v = Vector2D
            
              (x: 3.0, y: 4.0)
              
let w = Vector2D(x: 1.0, y: 2.0)
              

              
// Basic operations
              
let sum = v + w // Vector2D(x: 4.0, y: 6.0)
              
let scaled = 2.0 * v // Vector2D(x: 6.0, y: 8.0)
              

              
// Norm and distance
              
print(v.norm) // 5.0 (√(3² + 4²))
              
print(v.distance(to: w)) // 2.828…
              
print(v.dot(w)) // 11.0 (3
              1 + 42)
              

              
// 2D-specific operations
              
print(v.cross(w)) // 2.0 (pseudo-cross product)
              
print(v.angle) // 0.927… radians (~53°)
              
let rotated = v.rotated(by: .pi/2) // Vector2D(x: -4.0, y: 3.0)

Output:

5.0
2.8284271247461903
11.0
2.0
0.9272952180016122
Vector2D(x: -4.0, y: 3.0)

Vector3D: Fixed 3D Vectors

Use Cases:

Three-variable optimization
3D coordinate systems
RGB color spaces
Cross product calculations

Performance: Very fast (compile-time optimization)

import BusinessMath

// Create 3D vectors
let v3 = Vector3D
            
              (x: 1.0, y: 2.0, z: 3.0)
              
let w3 = Vector3D
              
                (x: 4.0, y: 5.0, z: 6.0)
                

                
// Basic operations
                
let sum3 = v3 + w3 // Vector3D(x: 5.0, y: 7.0, z: 9.0)
                
let scaled3 = 2.0 * v3 // Vector3D(x: 2.0, y: 4.0, z: 6.0)
                

                
// Norm and dot product
                
print(v3.norm) // 3.742… (√(1² + 2² + 3²))
                
print(v3.dot(w3)) // 32.0 (1
                4 + 25 + 3*6)
                

                
// 3D-specific: Cross product
                
let cross = v3.cross(w3) // Vector3D perpendicular to both
                
print(cross) // Vector3D(x: -3.0, y: 6.0, z: -3.0)
                

                
// Verify perpendicularity
                
print(v3.dot(cross)) // ~0.0 (perpendicular)
                
print(w3.dot(cross)) // ~0.0 (perpendicular)

Output:

3.7416573867739413
32.0
Vector3D(x: -3.0, y: 6.0, z: -3.0)
0.0
0.0

The insight: Cross product gives a vector perpendicular to both inputs—useful for 3D geometry and physics.

VectorN: Variable N-Dimensional Vectors

Use Cases:

High-dimensional optimization (N > 3)
Portfolio weights (N assets)
Machine learning feature vectors
Any variable or runtime-determined dimension

Performance: Flexible but has array bounds checking overhead

import BusinessMath

// Create an N-dimensional vector
let vN = VectorN
            
              ([1.0, 2.0, 3.0, 4.0, 5.0])
              
let wN = VectorN([5.0, 4.0, 3.0, 2.0, 1.0])
              

              
// Basic operations
              
let sumN = vN + wN // VectorN([6, 6, 6, 6, 6])
              
let scaledN = 2.0 * vN // VectorN([2, 4, 6, 8, 10])
              

              
// Norm and dot product
              
print(vN.norm) // 7.416… (√55)
              
print(vN.dot(wN)) // 35.0
              

              
// Element access
              
print(vN[0]) // 1.0
              
print(vN[2]) // 3.0
              

              
// Statistical operations
              
print(vN.dimension) // 5
              
print(vN.sum) // 15.0
              
print(vN.mean) // 3.0
              
print(vN.standardDeviation()) // 1.581…
              
print(vN.min) // 1.0
              
print(vN.max) // 5.0

Output:

7.416198487095663
35.0
1.0
3.0
5
15.0
3.0
1.5811388300841898
1.0
5.0

Common Operations

All vector types share these operations through the VectorSpace protocol:

Arithmetic Operations

let v = VectorN([1.0, 2.0, 3.0])
let w = VectorN([4.0, 5.0, 6.0])

// Addition and subtraction
let sum = v + w                    // [5, 7, 9]
let diff = v - w                   // [-3, -3, -3]

// Scalar multiplication
let scaled = 3.0 * v               // [3, 6, 9]
let divided = v / 2.0              // [0.5, 1.0, 1.5]

// Negation
let negated = -v                   // [-1, -2, -3]

Norms and Distances

let v = VectorN([3.0, 4.0])
let w = VectorN([0.0, 0.0])

// Euclidean norm
print(v.norm)                      // 5.0 (√(3² + 4²))
print(v.squaredNorm)               // 25.0 (faster for comparisons)

// Distance metrics
print(v.distance(to: w))           // 5.0 (Euclidean)
print(v.manhattanDistance(to: w))  // 7.0 (|3| + |4|)
print(v.chebyshevDistance(to: w))  // 4.0 (max(|3|, |4|))

Use cases:

Euclidean: Standard distance (geometric)
Manhattan: City-block distance (grids, taxi routes)
Chebyshev: Chessboard distance (king moves)

Dot Products and Angles

let v = VectorN([1.0, 0.0, 0.0])
let w = VectorN([0.0, 1.0, 0.0])

// Dot product
print(v.dot(w))                    // 0.0 (perpendicular)

// Cosine similarity
print(v.cosineSimilarity(with: w)) // 0.0 (orthogonal)

// Angle between vectors
let angle = v.angle(with: w)       // π/2 radians (90°)
print(angle * 180 / .pi)           // 90.0 degrees

Projections

let v = VectorN([3.0, 4.0])
let w = VectorN([1.0, 0.0])

// Project v onto w
let projection = v.projection(onto: w)  // [3.0, 0.0]

// Rejection (component perpendicular to w)
let rejection = v.rejection(from: w)    // [0.0, 4.0]

// Verify: v = projection + rejection
print(v == projection + rejection)      // true

Application: Decompose a vector into parallel and perpendicular components.

Normalization

let v = VectorN([3.0, 4.0])

// Normalize to unit length
let unit = v.normalized()          // [0.6, 0.8]
print(unit.norm)                   // 1.0

// Verify direction preserved
print(v.cosineSimilarity(with: unit))  // 1.0 (same direction)

Use case: Unit vectors for direction without magnitude.

VectorN-Specific Operations

Construction Methods

// From array
let v1 = VectorN([1.0, 2.0, 3.0])

// Repeating value
let v2 = VectorN(repeating: 5.0, count: 10)

// Zero vector
let v3 = VectorN
            
              .zero
              

              
// Ones vector
              
let v4 = VectorN
              
                .ones(dimension: 5)
                

                
// Basis vector (one component = 1, rest = 0)
                
let e2 = VectorN
                
                  .basisVector(dimension: 5, index: 2)
                  
// [0, 0, 1, 0, 0]
                  

                  
// Linear space (evenly spaced)
                  
let v5 = VectorN.linearSpace(from: 0.0, to: 10.0, count: 11)
                  
// [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
                  

                  
// Log space (logarithmically spaced)
                  
let v6 = VectorN.logSpace(from: 1.0, to: 100.0, count: 3)
                  
// [1, 10, 100]

Functional Operations

let v = VectorN([-2.0, -1.0, 0.0, 1.0, 2.0, 3.0])

// Map (element-wise transform)
let squared = v.map { $0 * $0 }    // [4, 1, 0, 1, 4, 9]

// Filter
let positive = v.filter { $0 > 0 } // [1, 2, 3]

// Reduce
let sum = v.reduce(0.0, +)         // 3.0

// Zip with another vector
let w = VectorN([4.0, 5.0, 6.0, 7.0, 8.0, 9.0])
let product = v.zipWith(w, *)      // [-8, -5, 0, 7, 16, 27]

Real-World Example: Portfolio Weights

import BusinessMath

// 4-asset portfolio
let assets = [“US Stocks”, “Intl Stocks”, “Bonds”, “Real Estate”]
let weights = VectorN([0.40, 0.25, 0.25, 0.10])
let expectedReturns = VectorN([0.10, 0.12, 0.04, 0.08])

// Verify fully invested (weights sum to 1.0)
print(“Fully invested: (weights.sum == 1.0)”)

// Portfolio expected return (weighted average)
let portfolioReturn = weights.dot(expectedReturns)
print(“Portfolio return: (portfolioReturn.percent(1))”)

// Normalize to equal weights for comparison
let equalWeights = VectorN
            
              .equalWeights(dimension: 4)
              
let equalReturn = equalWeights.dot(expectedReturns)
              
print(“Equal-weight return: (equalReturn.percent(1))”)

Output:

Fully invested: true
Portfolio return: 8.8%
Equal-weight return: 8.5%

Try It Yourself

Click to expand full playground code

import BusinessMath

// Create a 2D vector
let v = Vector2D
              
                (x: 3.0, y: 4.0)
                
let w = Vector2D(x: 1.0, y: 2.0)
                

                
// Basic operations
                
let sum = v + w // Vector2D(x: 4.0, y: 6.0)
                
let scaled = 2.0 * v // Vector2D(x: 6.0, y: 8.0)
                

                
// Norm and distance
                
print(v.norm) // 5.0 (√(3² + 4²))
                
print(v.distance(to: w)) // 2.828...
                
print(v.dot(w)) // 11.0 (3*1 + 4*2)
                

                
// 2D-specific operations
                
print(v.cross(w)) // 2.0 (pseudo-cross product)
                
print(v.angle) // 0.927... radians (~53°)
                
let rotated = v.rotated(by: .pi/2) // Vector2D(x: -4.0, y: 3.0)
                
print(rotated.toArray())
                

                
// MARK: Vector3D
                

                
 // Create 3D vectors
                
let v_3d = Vector3D
                
                  (x: 1.0, y: 2.0, z: 3.0)
                  
let w_3d = Vector3D
                  
                    (x: 4.0, y: 5.0, z: 6.0)
                    

                    
// Basic operations
                    
let sum3 = v_3d + w_3d // Vector3D(x: 5.0, y: 7.0, z: 9.0)
                    
let scaled3 = 2.0 * v_3d // Vector3D(x: 2.0, y: 4.0, z: 6.0)
                    

                    
// Norm and dot product
                    
print(v_3d.norm) // 3.742... (√(1² + 2² + 3²))
                    
print(v_3d.dot(w_3d)) // 32.0 (1*4 + 2*5 + 3*6)
                    

                    
// 3D-specific: Cross product
                    
let cross = v_3d.cross(w_3d) // Vector3D perpendicular to both
                    
print(cross) // Vector3D(x: -3.0, y: 6.0, z: -3.0)
                    

                    
// Verify perpendicularity
                    
print(v_3d.dot(cross)) // ~0.0 (perpendicular)
                    
print(w_3d.dot(cross)) // ~0.0 (perpendicular)
                    

                    
// MARK: VectorN
                    

                    
 // Create an N-dimensional vector
                    
 let vN = VectorN
                    
                      ([1.0, 2.0, 3.0, 4.0, 5.0])
                      
 let wN = VectorN([5.0, 4.0, 3.0, 2.0, 1.0])
                      

                      
 // Basic operations
                      
 let sumN = vN + wN // VectorN([6, 6, 6, 6, 6])
                      
 let scaledN = 2.0 * vN // VectorN([2, 4, 6, 8, 10])
                      

                      
 // Norm and dot product
                      
 print(vN.norm) // 7.416... (√55)
                      
 print(vN.dot(wN)) // 35.0
                      

                      
 // Element access
                      
 print(vN[0]) // 1.0
                      
 print(vN[2]) // 3.0
                      

                      
 // Statistical operations
                      
 print(vN.dimension) // 5
                      
 print(vN.sum) // 15.0
                      
 print(vN.mean) // 3.0
                      
 print(vN.standardDeviation()) // 1.581...
                      
 print(vN.min) // 1.0
                      
 print(vN.max) // 5.0
                      

                      
// MARK: - Arithmetic Operations
                      

                      
let v_arith = VectorN([1.0, 2.0, 3.0])
                      
let w_arith = VectorN([4.0, 5.0, 6.0])
                      

                      
// Addition and subtraction
                      
let sum_arith = v_arith + w_arith // [5, 7, 9]
                      
let diff_arith = v_arith - w_arith // [-3, -3, -3]
                      

                      
// Scalar multiplication
                      
let scaled_arith = 3.0 * v_arith // [3, 6, 9]
                      
let divided = v_arith / 2.0 // [0.5, 1.0, 1.5]
                      

                      
// Negation
                      
let negated = -v_arith // [-1, -2, -3]
                      

                      
// MARK: - Norms and Distances
                      

                      
let v_norm = VectorN([3.0, 4.0])
                      
let w_norm = VectorN([0.0, 0.0])
                      

                      
// Euclidean norm
                      
print(v_norm.norm) // 5.0 (√(3² + 4²))
                      
print(v_norm.squaredNorm) // 25.0 (faster for comparisons)
                      

                      
// Distance metrics
                      
print(v_norm.distance(to: w_norm)) // 5.0 (Euclidean)
                      
print(v_norm.manhattanDistance(to: w_norm)) // 7.0 (|3| + |4|)
                      
print(v_norm.chebyshevDistance(to: w_norm)) // 4.0 (max(|3|, |4|))
                      

                      

                      
// MARK: - Dot Products and Angles
                      

                      
let v_dot = VectorN([1.0, 0.0, 0.0])
                      
let w_dot = VectorN([0.0, 1.0, 0.0])
                      

                      
// Dot product
                      
print(v_dot.dot(w_dot)) // 0.0 (perpendicular)
                      

                      
// Cosine similarity
                      
print(v_dot.cosineSimilarity(with: w_dot)) // 0.0 (orthogonal)
                      

                      
// Angle between vectors
                      
let angle_dot = v_dot.angle(with: w_dot) // π/2 radians (90°)
                      
print(angle_dot * 180 / .pi) // 90.0 degrees
                      

                      

                      
// MARK: Projections
                      

                      
let v_proj = VectorN([3.0, 4.0])
                      
let w_proj = VectorN([1.0, 0.0])
                      

                      
// Project v onto w
                      
let projection = v_proj.projection(onto: w_proj) // [3.0, 0.0]
                      

                      
// Rejection (component perpendicular to w)
                      
let rejection = v_proj.rejection(from: w_proj) // [0.0, 4.0]
                      

                      
// Verify: v = projection + rejection
                      
print(v_proj == projection + rejection) // true
                      

                      
// MARK: - Normalization
                      

                      

                      

                      
// Normalize to unit length
                      
let unit = v_norm.normalized() // [0.6, 0.8]
                      
print(unit.norm) // 1.0
                      

                      
// Verify direction preserved
                      
print(v_norm.cosineSimilarity(with: unit)) // 1.0 (same direction)
                      

                      

                      
// MARK: - VectorN Specific Construction
                      

                      
 // From array
                      
 let v1 = VectorN([1.0, 2.0, 3.0])
                      

                      
 // Repeating value
                      
 let v2 = VectorN(repeating: 5.0, count: 10)
                      

                      
 // Zero vector
                      
 let v3 = VectorN
                      
                        .zero
                        

                        
 // Ones vector
                        
 let v4 = VectorN
                        
                          .ones(dimension: 5)
                          

                          
 // Basis vector (one component = 1, rest = 0)
                          
 let e2 = VectorN
                          
                            .basisVector(dimension: 5, index: 2)
                            
 // [0, 0, 1, 0, 0]
                            

                            
 // Linear space (evenly spaced)
                            
 let v5 = VectorN.linearSpace(from: 0.0, to: 10.0, count: 11)
                            
 // [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
                            

                            
 // Log space (logarithmically spaced)
                            
 let v6 = VectorN.logSpace(from: 1.0, to: 100.0, count: 3)
                            
 // [1, 10, 100]
                            

                            
// MARK: - Functional Operations
                            

                            
let v_func = VectorN([-2.0, -1.0, 0.0, 1.0, 2.0, 3.0])
                            

                            
// Map (element-wise transform)
                            
let squared_func = v_func.map { $0 * $0 } // [4, 1, 0, 1, 4, 9]
                            

                            
// Filter
                            
let positive_func = v_func.filter { $0 > 0 } // [1, 2, 3]
                            

                            
// Reduce
                            
let sum_func = v_func.reduce(0.0, +) // 3.0
                            

                            
// Zip with another vector
                            
let w_func = VectorN([4.0, 5.0, 6.0, 7.0, 8.0, 9.0])
                            
let product_func = v_func.zipWith(w_func, *) // [-8, -5, 0, 7, 16, 27]
                            
print(product_func)
                            

                            

                            
// MARK: Portfolio Weights Example
                            

                            
// 4-asset portfolio
                            
let assets = ["US Stocks", "Intl Stocks", "Bonds", "Real Estate"]
                            
let weights = VectorN([0.40, 0.25, 0.25, 0.10])
                            
let expectedReturns = VectorN([0.10, 0.12, 0.04, 0.08])
                            

                            
// Verify fully invested (weights sum to 1.0)
                            
print("Fully invested: \(weights.sum == 1.0)")
                            

                            
// Portfolio expected return (weighted average)
                            
let portfolioReturn = weights.dot(expectedReturns)
                            
print("Portfolio return: \(portfolioReturn.percent(1))")
                            

                            
// Equal weights for comparison (each asset gets 25%)
                            
let equalWeights = VectorN
                            
                              .equalWeights(dimension: 4)
                              
print("Equal weights: \(equalWeights.toArray())") // [0.25, 0.25, 0.25, 0.25]
                              
print("Sum: \(equalWeights.sum)") // 1.0
                              
let equalReturn = equalWeights.dot(expectedReturns)
                              
print("Equal-weight return: \(equalReturn.percent(1))") // 8.5%
                              

                              
// MARK: - Simplex Projection vs Normalization
                              

                              
// Demonstrate the difference between simplex projection and normalization
                              
let rawScores = VectorN([3.0, 1.0, 2.0])
                              

                              
// Simplex projection: components sum to 1.0
                              
let probabilities = rawScores.simplexProjection()
                              
print("\nSimplex projection (sum = 1.0):")
                              
print(" Values: \(probabilities.toArray().map { $0.number(3) })")
                              
print(" Sum: \(probabilities.sum.number(2))")
                              
print(" Norm: \(probabilities.norm.number(3))")
                              

                              
// Normalization: Euclidean norm = 1.0
                              
let unitVector = rawScores.normalized()
                              
print("\nNormalization (norm = 1.0):")
                              
print(" Values: \(unitVector.toArray().map { $0.number(3) })")
                              
print(" Sum: \(unitVector.sum.number(3))")
                              
print(" Norm: \(unitVector.norm.number(2))")

→ Full API Reference: BusinessMath Docs – 5.4 Vector Operations

Modifications to try:

Build a 10-asset portfolio and compute risk contribution per asset
Use cross product to compute area of triangle (3D vectors)
Implement Gram-Schmidt orthogonalization using projections
Compare performance: Vector2D vs. VectorN for 2D optimization

Real-World Application

Portfolio management: Represent asset allocations as vectors
Machine learning: Feature vectors, gradient descent
Engineering: Force vectors, velocity vectors, state spaces
Optimization: Multivariate parameter spaces

Data scientist use case: “I need to optimize hyperparameters for a model with 20 features. The algorithm should work whether I have 2 features or 200.”

Generic vector operations make this trivial.

★ Insight ─────────────────────────────────────

Why Dot Product Measures Similarity

The dot product v · w = ‖v‖ ‖w‖ cos(θ) combines magnitude and angle.

Cosine similarity normalizes out magnitude: cos(θ) = (v · w) / (‖v‖ ‖w‖)

Interpretation:

cos(θ) = 1: Same direction (parallel)
cos(θ) = 0: Perpendicular (orthogonal)
cos(θ) = -1: Opposite direction (antiparallel)

Application - Portfolio correlation: If returns for two assets are vectors over time, their cosine similarity measures correlation. High similarity means they move together (bad for diversification).

Rule of thumb: Maximize portfolio diversity = minimize pairwise cosine similarity.

─────────────────────────────────────────────────

📝 Development Note

The hardest design decision was choosing the right vector protocol hierarchy. We considered:

Single protocol (what we chose): VectorSpace with all operations
Layered protocols: VectorAddition, VectorNorm, VectorDot
Class hierarchy: AbstractVector base class

We chose single protocol because:

Swift favors protocol composition over class inheritance
All vector operations need all capabilities (no partial implementations)
Generic constraints are simpler: vs.

Trade-off: Implementing VectorSpace requires all methods. But this ensures every vector type is fully functional.

Related Methodology: Protocol-Oriented Design (Week 1) - Covered protocol composition and generic programming.

Next Steps

Coming up next week: Advanced Optimization (Week 8) - Multivariate Newton-Raphson, constrained optimization with Lagrange multipliers, and a portfolio optimization case study.

Series Progress:

Week: 7/12
Posts Published: 25/~48
Playgrounds: 21 available

Tagged with: swift-patterns