Vector Operations: Foundation for Multivariate Optimization

BusinessMath Quarterly Series

14 min read

Part 25 of 12-Week BusinessMath Series

What You’ll Learn

  • Understanding the VectorSpace protocol and why it matters
  • Working with Vector2D, Vector3D, and VectorN types
  • Performing vector operations: norms, dot products, projections
  • Using generic algorithms that work across all vector types
  • Creating type-safe multivariate constraints
  • Building portfolio weights and feature vectors

The Problem

Multivariate optimization requires working with vectors of different dimensions:

  • Portfolio optimization: N-dimensional weights for N assets
  • Pricing models: 2D or 3D parameter spaces
  • Machine learning: High-dimensional feature vectors
  • Generic algorithms: Code that works for any dimension

Without a unified vector abstraction, you’d write duplicate code for each dimension (optimize2D, optimize3D, optimizeND, etc.).

The Solution

BusinessMath’s VectorSpace protocol provides a generic interface for vector operations. Write optimization algorithms once, they work for all dimensions.

The VectorSpace Protocol

A vector space is a mathematical structure supporting:

  • Vector addition: v + w
  • Scalar multiplication: α · v
  • Zero element: 0
  • Norms and distances: ‖v‖, ‖v - w‖

Protocol Definition (simplified):

public protocol VectorSpace: AdditiveArithmetic {
    associatedtype Scalar: Real

    // Required operations
    static var zero: Self { get }
    static func + (lhs: Self, rhs: Self) -> Self
    static func * (lhs: Scalar, rhs: Self) -> Self

    // Norm and distance
    var norm: Scalar { get }
    func dot(_ other: Self) -> Scalar

    // Conversion
    static func fromArray(_ array: [Scalar]) -> Self?
    func toArray() -> [Scalar]
}

Why it matters:

// ❌ Before: Duplicate implementations
func optimize2D(_ f: (Vector2D) -> Double, ...) -> Vector2D
func optimize3D(_ f: (Vector3D) -> Double, ...) -> Vector3D
func optimizeND(_ f: (VectorN) -> Double, ...) -> VectorN

// ✅ After: One generic implementation
func optimize(_ f: (V) -> V.Scalar, ...) -> V

One algorithm works for all vector types!

Vector Implementations

BusinessMath provides three vector types optimized for different use cases.

Vector2D: Fixed 2D Vectors

Use Cases:

  • Two-variable optimization
  • Coordinate systems (x, y)
  • Complex numbers (real, imaginary)

Performance: Fastest (compile-time optimization, zero array overhead)

import BusinessMath

// Create a 2D vector
let v = Vector2D(x: 3.0, y: 4.0)
let w = Vector2D(x: 1.0, y: 2.0)

// Basic operations
let sum = v + w                    // Vector2D(x: 4.0, y: 6.0)
let scaled = 2.0 * v               // Vector2D(x: 6.0, y: 8.0)

// Norm and distance
print(v.norm)                      // 5.0 (√(3² + 4²))
print(v.distance(to: w))           // 2.828...
print(v.dot(w))                    // 11.0 (3*1 + 4*2)

// 2D-specific operations
print(v.cross(w))                  // 2.0 (pseudo-cross product)
print(v.angle)                     // 0.927... radians (~53°)
let rotated = v.rotated(by: .pi/2) // Vector2D(x: -4.0, y: 3.0)

Output:

5.0
2.8284271247461903
11.0
2.0
0.9272952180016122
Vector2D(x: -4.0, y: 3.0)

Vector3D: Fixed 3D Vectors

Use Cases:

  • Three-variable optimization
  • 3D coordinate systems
  • RGB color spaces
  • Cross product calculations

Performance: Very fast (compile-time optimization)

import BusinessMath

// Create 3D vectors
let v3 = Vector3D(x: 1.0, y: 2.0, z: 3.0)
let w3 = Vector3D(x: 4.0, y: 5.0, z: 6.0)

// Basic operations
let sum3 = v3 + w3                 // Vector3D(x: 5.0, y: 7.0, z: 9.0)
let scaled3 = 2.0 * v3             // Vector3D(x: 2.0, y: 4.0, z: 6.0)

// Norm and dot product
print(v3.norm)                     // 3.742... (√(1² + 2² + 3²))
print(v3.dot(w3))                  // 32.0 (1*4 + 2*5 + 3*6)

// 3D-specific: Cross product
let cross = v3.cross(w3)           // Vector3D perpendicular to both
print(cross)                       // Vector3D(x: -3.0, y: 6.0, z: -3.0)

// Verify perpendicularity
print(v3.dot(cross))               // ~0.0 (perpendicular)
print(w3.dot(cross))               // ~0.0 (perpendicular)

Output:

3.7416573867739413
32.0
Vector3D(x: -3.0, y: 6.0, z: -3.0)
0.0
0.0

The insight: Cross product gives a vector perpendicular to both inputs—useful for 3D geometry and physics.

VectorN: Variable N-Dimensional Vectors

Use Cases:

  • High-dimensional optimization (N > 3)
  • Portfolio weights (N assets)
  • Machine learning feature vectors
  • Any variable or runtime-determined dimension

Performance: Flexible but has array bounds checking overhead

import BusinessMath

// Create an N-dimensional vector
let vN = VectorN([1.0, 2.0, 3.0, 4.0, 5.0])
let wN = VectorN([5.0, 4.0, 3.0, 2.0, 1.0])

// Basic operations
let sumN = vN + wN                 // VectorN([6, 6, 6, 6, 6])
let scaledN = 2.0 * vN             // VectorN([2, 4, 6, 8, 10])

// Norm and dot product
print(vN.norm)                     // 7.416... (√55)
print(vN.dot(wN))                  // 35.0

// Element access
print(vN[0])                       // 1.0
print(vN[2])                       // 3.0

// Statistical operations
print(vN.dimension)                // 5
print(vN.sum)                      // 15.0
print(vN.mean)                     // 3.0
print(vN.standardDeviation())      // 1.581...
print(vN.min)                      // 1.0
print(vN.max)                      // 5.0

Output:

7.416198487095663
35.0
1.0
3.0
5
15.0
3.0
1.5811388300841898
1.0
5.0

Common Operations

All vector types share these operations through the VectorSpace protocol:

Arithmetic Operations

let v = VectorN([1.0, 2.0, 3.0])
let w = VectorN([4.0, 5.0, 6.0])

// Addition and subtraction
let sum = v + w                    // [5, 7, 9]
let diff = v - w                   // [-3, -3, -3]

// Scalar multiplication
let scaled = 3.0 * v               // [3, 6, 9]
let divided = v / 2.0              // [0.5, 1.0, 1.5]

// Negation
let negated = -v                   // [-1, -2, -3]

Norms and Distances

let v = VectorN([3.0, 4.0])
let w = VectorN([0.0, 0.0])

// Euclidean norm
print(v.norm)                      // 5.0 (√(3² + 4²))
print(v.squaredNorm)               // 25.0 (faster for comparisons)

// Distance metrics
print(v.distance(to: w))           // 5.0 (Euclidean)
print(v.manhattanDistance(to: w))  // 7.0 (|3| + |4|)
print(v.chebyshevDistance(to: w))  // 4.0 (max(|3|, |4|))

Use cases:

  • Euclidean: Standard distance (geometric)
  • Manhattan: City-block distance (grids, taxi routes)
  • Chebyshev: Chessboard distance (king moves)

Dot Products and Angles

let v = VectorN([1.0, 0.0, 0.0])
let w = VectorN([0.0, 1.0, 0.0])

// Dot product
print(v.dot(w))                    // 0.0 (perpendicular)

// Cosine similarity
print(v.cosineSimilarity(with: w)) // 0.0 (orthogonal)

// Angle between vectors
let angle = v.angle(with: w)       // π/2 radians (90°)
print(angle * 180 / .pi)           // 90.0 degrees

Projections

let v = VectorN([3.0, 4.0])
let w = VectorN([1.0, 0.0])

// Project v onto w
let projection = v.projection(onto: w)  // [3.0, 0.0]

// Rejection (component perpendicular to w)
let rejection = v.rejection(from: w)    // [0.0, 4.0]

// Verify: v = projection + rejection
print(v == projection + rejection)      // true

Application: Decompose a vector into parallel and perpendicular components.

Normalization

let v = VectorN([3.0, 4.0])

// Normalize to unit length
let unit = v.normalized()          // [0.6, 0.8]
print(unit.norm)                   // 1.0

// Verify direction preserved
print(v.cosineSimilarity(with: unit))  // 1.0 (same direction)

Use case: Unit vectors for direction without magnitude.

VectorN-Specific Operations

Construction Methods

// From array
let v1 = VectorN([1.0, 2.0, 3.0])

// Repeating value
let v2 = VectorN(repeating: 5.0, count: 10)

// Zero vector
let v3 = VectorN.zero

// Ones vector
let v4 = VectorN.ones(dimension: 5)

// Basis vector (one component = 1, rest = 0)
let e2 = VectorN.basisVector(dimension: 5, index: 2)
// [0, 0, 1, 0, 0]

// Linear space (evenly spaced)
let v5 = VectorN.linearSpace(from: 0.0, to: 10.0, count: 11)
// [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

// Log space (logarithmically spaced)
let v6 = VectorN.logSpace(from: 1.0, to: 100.0, count: 3)
// [1, 10, 100]

Functional Operations

let v = VectorN([-2.0, -1.0, 0.0, 1.0, 2.0, 3.0])

// Map (element-wise transform)
let squared = v.map { $0 * $0 }    // [4, 1, 0, 1, 4, 9]

// Filter
let positive = v.filter { $0 > 0 } // [1, 2, 3]

// Reduce
let sum = v.reduce(0.0, +)         // 3.0

// Zip with another vector
let w = VectorN([4.0, 5.0, 6.0, 7.0, 8.0, 9.0])
let product = v.zipWith(w, *)      // [-8, -5, 0, 7, 16, 27]

Real-World Example: Portfolio Weights

import BusinessMath

// 4-asset portfolio
let assets = ["US Stocks", "Intl Stocks", "Bonds", "Real Estate"]
let weights = VectorN([0.40, 0.25, 0.25, 0.10])
let expectedReturns = VectorN([0.10, 0.12, 0.04, 0.08])

// Verify fully invested (weights sum to 1.0)
print("Fully invested: \(weights.sum == 1.0)")

// Portfolio expected return (weighted average)
let portfolioReturn = weights.dot(expectedReturns)
print("Portfolio return: \(portfolioReturn.percent(1))")

// Normalize to equal weights for comparison
let equalWeights = VectorN.equalWeights(dimension: 4)
let equalReturn = equalWeights.dot(expectedReturns)
print("Equal-weight return: \(equalReturn.percent(1))")

Output:

Fully invested: true
Portfolio return: 8.8%
Equal-weight return: 8.5%

Try It Yourself

Click to expand full playground code 
import BusinessMath

// Create a 2D vector
let v = Vector2D(x: 3.0, y: 4.0)
let w = Vector2D(x: 1.0, y: 2.0)

// Basic operations
let sum = v + w                    // Vector2D(x: 4.0, y: 6.0)
let scaled = 2.0 * v               // Vector2D(x: 6.0, y: 8.0)

// Norm and distance
print(v.norm)                      // 5.0 (√(3² + 4²))
print(v.distance(to: w))           // 2.828...
print(v.dot(w))                    // 11.0 (3*1 + 4*2)

// 2D-specific operations
print(v.cross(w))                  // 2.0 (pseudo-cross product)
print(v.angle)                     // 0.927... radians (~53°)
let rotated = v.rotated(by: .pi/2) // Vector2D(x: -4.0, y: 3.0)
print(rotated.toArray())

// MARK: Vector3D

	// Create 3D vectors
let v_3d = Vector3D(x: 1.0, y: 2.0, z: 3.0)
let w_3d = Vector3D(x: 4.0, y: 5.0, z: 6.0)

// Basic operations
let sum3 = v_3d + w_3d                 // Vector3D(x: 5.0, y: 7.0, z: 9.0)
let scaled3 = 2.0 * v_3d             // Vector3D(x: 2.0, y: 4.0, z: 6.0)

// Norm and dot product
print(v_3d.norm)                     // 3.742... (√(1² + 2² + 3²))
print(v_3d.dot(w_3d))                  // 32.0 (1*4 + 2*5 + 3*6)

// 3D-specific: Cross product
let cross = v_3d.cross(w_3d)           // Vector3D perpendicular to both
print(cross)                       // Vector3D(x: -3.0, y: 6.0, z: -3.0)

// Verify perpendicularity
print(v_3d.dot(cross))               // ~0.0 (perpendicular)
print(w_3d.dot(cross))               // ~0.0 (perpendicular)

// MARK: VectorN

	// Create an N-dimensional vector
	let vN = VectorN([1.0, 2.0, 3.0, 4.0, 5.0])
	let wN = VectorN([5.0, 4.0, 3.0, 2.0, 1.0])

	// Basic operations
	let sumN = vN + wN                 // VectorN([6, 6, 6, 6, 6])
	let scaledN = 2.0 * vN             // VectorN([2, 4, 6, 8, 10])

	// Norm and dot product
	print(vN.norm)                     // 7.416... (√55)
	print(vN.dot(wN))                  // 35.0

	// Element access
	print(vN[0])                       // 1.0
	print(vN[2])                       // 3.0

	// Statistical operations
	print(vN.dimension)                // 5
	print(vN.sum)                      // 15.0
	print(vN.mean)                     // 3.0
	print(vN.standardDeviation())      // 1.581...
	print(vN.min)                      // 1.0
	print(vN.max)                      // 5.0

// MARK: - Arithmetic Operations

let v_arith = VectorN([1.0, 2.0, 3.0])
let w_arith = VectorN([4.0, 5.0, 6.0])

// Addition and subtraction
let sum_arith = v_arith + w_arith                    // [5, 7, 9]
let diff_arith = v_arith - w_arith                   // [-3, -3, -3]

// Scalar multiplication
let scaled_arith = 3.0 * v_arith               // [3, 6, 9]
let divided = v_arith / 2.0              // [0.5, 1.0, 1.5]

// Negation
let negated = -v_arith                   // [-1, -2, -3]

// MARK: - Norms and Distances

let v_norm = VectorN([3.0, 4.0])
let w_norm = VectorN([0.0, 0.0])

// Euclidean norm
print(v_norm.norm)                      // 5.0 (√(3² + 4²))
print(v_norm.squaredNorm)               // 25.0 (faster for comparisons)

// Distance metrics
print(v_norm.distance(to: w_norm))           // 5.0 (Euclidean)
print(v_norm.manhattanDistance(to: w_norm))  // 7.0 (|3| + |4|)
print(v_norm.chebyshevDistance(to: w_norm))  // 4.0 (max(|3|, |4|))


// MARK: - Dot Products and Angles

let v_dot = VectorN([1.0, 0.0, 0.0])
let w_dot = VectorN([0.0, 1.0, 0.0])

// Dot product
print(v_dot.dot(w_dot))                    // 0.0 (perpendicular)

// Cosine similarity
print(v_dot.cosineSimilarity(with: w_dot)) // 0.0 (orthogonal)

// Angle between vectors
let angle_dot = v_dot.angle(with: w_dot)       // π/2 radians (90°)
print(angle_dot * 180 / .pi)           // 90.0 degrees


// MARK: Projections

let v_proj = VectorN([3.0, 4.0])
let w_proj = VectorN([1.0, 0.0])

// Project v onto w
let projection = v_proj.projection(onto: w_proj)  // [3.0, 0.0]

// Rejection (component perpendicular to w)
let rejection = v_proj.rejection(from: w_proj)    // [0.0, 4.0]

// Verify: v = projection + rejection
print(v_proj == projection + rejection)      // true

// MARK: - Normalization



// Normalize to unit length
let unit = v_norm.normalized()          // [0.6, 0.8]
print(unit.norm)                   // 1.0

// Verify direction preserved
print(v_norm.cosineSimilarity(with: unit))  // 1.0 (same direction)


// MARK: - VectorN Specific Construction

	// From array
	let v1 = VectorN([1.0, 2.0, 3.0])

	// Repeating value
	let v2 = VectorN(repeating: 5.0, count: 10)

	// Zero vector
	let v3 = VectorN.zero

	// Ones vector
	let v4 = VectorN.ones(dimension: 5)

	// Basis vector (one component = 1, rest = 0)
	let e2 = VectorN.basisVector(dimension: 5, index: 2)
	// [0, 0, 1, 0, 0]

	// Linear space (evenly spaced)
	let v5 = VectorN.linearSpace(from: 0.0, to: 10.0, count: 11)
	// [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

	// Log space (logarithmically spaced)
	let v6 = VectorN.logSpace(from: 1.0, to: 100.0, count: 3)
	// [1, 10, 100]

// MARK: - Functional Operations

let v_func = VectorN([-2.0, -1.0, 0.0, 1.0, 2.0, 3.0])

// Map (element-wise transform)
let squared_func = v_func.map { $0 * $0 }    // [4, 1, 0, 1, 4, 9]

// Filter
let positive_func = v_func.filter { $0 > 0 } // [1, 2, 3]

// Reduce
let sum_func = v_func.reduce(0.0, +)         // 3.0

// Zip with another vector
let w_func = VectorN([4.0, 5.0, 6.0, 7.0, 8.0, 9.0])
let product_func = v_func.zipWith(w_func, *)      // [-8, -5, 0, 7, 16, 27]
print(product_func)


// MARK: Portfolio Weights Example

// 4-asset portfolio
let assets = ["US Stocks", "Intl Stocks", "Bonds", "Real Estate"]
let weights = VectorN([0.40, 0.25, 0.25, 0.10])
let expectedReturns = VectorN([0.10, 0.12, 0.04, 0.08])

// Verify fully invested (weights sum to 1.0)
print("Fully invested: \(weights.sum == 1.0)")

// Portfolio expected return (weighted average)
let portfolioReturn = weights.dot(expectedReturns)
print("Portfolio return: \(portfolioReturn.percent(1))")

// Equal weights for comparison (each asset gets 25%)
let equalWeights = VectorN.equalWeights(dimension: 4)
print("Equal weights: \(equalWeights.toArray())")  // [0.25, 0.25, 0.25, 0.25]
print("Sum: \(equalWeights.sum)")  // 1.0
let equalReturn = equalWeights.dot(expectedReturns)
print("Equal-weight return: \(equalReturn.percent(1))")  // 8.5%

// MARK: - Simplex Projection vs Normalization

// Demonstrate the difference between simplex projection and normalization
let rawScores = VectorN([3.0, 1.0, 2.0])

// Simplex projection: components sum to 1.0
let probabilities = rawScores.simplexProjection()
print("\nSimplex projection (sum = 1.0):")
print("  Values: \(probabilities.toArray().map { $0.number(3) })")
print("  Sum: \(probabilities.sum.number(2))")
print("  Norm: \(probabilities.norm.number(3))")

// Normalization: Euclidean norm = 1.0
let unitVector = rawScores.normalized()
print("\nNormalization (norm = 1.0):")
print("  Values: \(unitVector.toArray().map { $0.number(3) })")
print("  Sum: \(unitVector.sum.number(3))")
print("  Norm: \(unitVector.norm.number(2))")

→ Full API Reference: BusinessMath Docs – 5.4 Vector Operations

Modifications to try:

  1. Build a 10-asset portfolio and compute risk contribution per asset
  2. Use cross product to compute area of triangle (3D vectors)
  3. Implement Gram-Schmidt orthogonalization using projections
  4. Compare performance: Vector2D vs. VectorN for 2D optimization

Real-World Application

  • Portfolio management: Represent asset allocations as vectors
  • Machine learning: Feature vectors, gradient descent
  • Engineering: Force vectors, velocity vectors, state spaces
  • Optimization: Multivariate parameter spaces

Data scientist use case: “I need to optimize hyperparameters for a model with 20 features. The algorithm should work whether I have 2 features or 200.”

Generic vector operations make this trivial.

★ Insight ─────────────────────────────────────

Why Dot Product Measures Similarity

The dot product v · w = ‖v‖ ‖w‖ cos(θ) combines magnitude and angle.

Cosine similarity normalizes out magnitude: cos(θ) = (v · w) / (‖v‖ ‖w‖)

Interpretation:

  • cos(θ) = 1: Same direction (parallel)
  • cos(θ) = 0: Perpendicular (orthogonal)
  • cos(θ) = -1: Opposite direction (antiparallel)

Application - Portfolio correlation:If returns for two assets are vectors over time, their cosine similarity measures correlation. High similarity means they move together (bad for diversification).

Rule of thumb: Maximize portfolio diversity = minimize pairwise cosine similarity.

─────────────────────────────────────────────────

📝 Development Note

The hardest design decision was choosing the right vector protocol hierarchy. We considered:

  1. Single protocol (what we chose): VectorSpace with all operations
  2. Layered protocols: VectorAddition, VectorNorm, VectorDot
  3. Class hierarchy: AbstractVector base class

We chose single protocol because:

  • Swift favors protocol composition over class inheritance
  • All vector operations need all capabilities (no partial implementations)
  • Generic constraints are simpler: vs.

Trade-off: Implementing VectorSpace requires all methods. But this ensures every vector type is fully functional.

Related Methodology: Protocol-Oriented Design (Week 1) - Covered protocol composition and generic programming.

Next Steps

Coming up next week: Advanced Optimization (Week 8) - Multivariate Newton-Raphson, constrained optimization with Lagrange multipliers, and a portfolio optimization case study.

Series Progress:

  • Week: 7/12
  • Posts Published: 25/~48
  • Playgrounds: 21 available

Tagged with: businessmath, swift, vectors, vectorspace, linear-algebra, protocols, generics