Amazon Web Services Analysis
Technical analysis of Amazon Web Services capabilities for hosting network boot infrastructure
This section contains detailed analysis of Amazon Web Services (AWS) for hosting the network boot server infrastructure, evaluating its support for TFTP, HTTP/HTTPS routing, and WireGuard VPN connectivity as required by ADR-0002.
Overview
Amazon Web Services is Amazon’s comprehensive cloud computing platform, offering compute, storage, networking, and managed services. This analysis focuses on AWS’s capabilities to support the network boot architecture decided in ADR-0002.
Key Services Evaluated
- EC2: Virtual machine instances for hosting boot server
- VPN / VPC: Network connectivity and VPN capabilities
- Elastic Load Balancing: Application and Network Load Balancers
- NAT Gateway: Network address translation for outbound connectivity
- VPC: Virtual Private Cloud networking and routing
Documentation Sections
1 - AWS Network Boot Protocol Support
Analysis of Amazon Web Services support for TFTP, HTTP, and HTTPS routing for network boot infrastructure
Network Boot Protocol Support on Amazon Web Services
This document analyzes AWS’s capabilities for hosting network boot infrastructure, specifically focusing on TFTP, HTTP, and HTTPS protocol support.
TFTP (Trivial File Transfer Protocol) Support
Native Support
Status: ❌ Not natively supported by Elastic Load Balancing
AWS’s Elastic Load Balancing services do not support TFTP protocol natively:
- Application Load Balancer (ALB): HTTP/HTTPS only (Layer 7)
- Network Load Balancer (NLB): TCP/UDP support, but not TFTP-aware
- Classic Load Balancer: Deprecated, similar limitations
TFTP operates on UDP port 69 with unique protocol semantics (variable block sizes, retransmissions, port negotiation) that standard load balancers cannot parse.
Implementation Options
Option 1: Direct EC2 Instance Access (Recommended for VPN Scenario)
Since ADR-0002 specifies a VPN-based architecture, TFTP can be served directly from an EC2 instance:
- Approach: Run TFTP server (e.g.,
tftpd-hpa, dnsmasq) on an EC2 instance - Access: Home lab connects via VPN tunnel to instance’s private IP
- Security Group: Allow UDP/69 from VPN subnet/security group
- Pros:
- Simple implementation
- No load balancer needed (single boot server sufficient for home lab)
- TFTP traffic encrypted through VPN tunnel
- Direct instance-to-client communication
- Cons:
- Single point of failure (no HA)
- Manual failover if instance fails
Option 2: Network Load Balancer (NLB) UDP Passthrough
While NLB doesn’t understand TFTP protocol, it can forward UDP traffic:
- Approach: Configure NLB to forward UDP/69 to target group
- Limitations:
- No TFTP-specific health checks
- Health checks would use TCP or different protocol
- Adds cost and complexity without significant benefit for single server
- Use Case: Only relevant for multi-AZ HA deployment (overkill for home lab)
TFTP Security Considerations
- Encryption: TFTP itself is unencrypted, but VPN tunnel provides encryption
- Security Groups: Restrict UDP/69 to VPN security group or CIDR only
- File Access Control: Configure TFTP server with restricted file access
- Read-Only Mode: Deploy TFTP server in read-only mode to prevent uploads
HTTP Support
Native Support
Status: ✅ Fully supported
AWS provides comprehensive HTTP support through multiple services:
Elastic Load Balancing - Application Load Balancer
- Protocol Support: HTTP/1.1, HTTP/2, HTTP/3 (preview)
- Port: Any port (typically 80 for HTTP)
- Routing: Path-based, host-based, query string, header-based routing
- Health Checks: HTTP health checks with configurable paths and response codes
- SSL Offloading: Terminate SSL at ALB and use HTTP to backend
- Backend: EC2 instances, ECS, EKS, Lambda
EC2 Direct Access
For VPN scenario, HTTP can be served directly from EC2 instance:
- Approach: Run HTTP server (nginx, Apache, custom service) on EC2
- Access: Home lab accesses via VPN tunnel to private IP
- Security Group: Allow TCP/80 from VPN security group
- Pros: Simpler than ALB for single boot server
HTTP Boot Flow for Network Boot
- PXE → TFTP: Initial bootloader (iPXE) loaded via TFTP
- iPXE → HTTP: iPXE chainloads kernel/initrd via HTTP
- Kernel/Initrd: Large boot files served efficiently over HTTP
- Connection Pooling: HTTP/1.1 keep-alive reduces connection overhead
- Compression: gzip compression for text-based configs
- CloudFront: Optional CDN for caching boot files (probably overkill for VPN scenario)
- TCP Optimization: AWS network optimized for low-latency TCP
HTTPS Support
Native Support
Status: ✅ Fully supported with advanced features
AWS provides enterprise-grade HTTPS support:
Elastic Load Balancing - Application Load Balancer
- Protocol Support: HTTPS/1.1, HTTP/2 over TLS, HTTP/3 (preview)
- SSL/TLS Termination: Terminate SSL at ALB
- Certificate Management:
- AWS Certificate Manager (ACM) - free SSL certificates with automatic renewal
- Import custom certificates
- Integration with private CA via ACM Private CA
- TLS Versions: TLS 1.0, 1.1, 1.2, 1.3 (configurable via security policy)
- Cipher Suites: Predefined security policies (modern, compatible, legacy)
- SNI Support: Multiple certificates on single load balancer
AWS Certificate Manager (ACM)
- Free Certificates: No cost for public SSL certificates used with AWS services
- Automatic Renewal: ACM automatically renews certificates before expiration
- Private CA: ACM Private CA for internal PKI (additional cost)
- Integration: Native integration with ALB, CloudFront, API Gateway
HTTPS for Network Boot
Use Case
Modern UEFI firmware and iPXE support HTTPS boot:
- iPXE HTTPS: iPXE compiled with
DOWNLOAD_PROTO_HTTPS can fetch over HTTPS - UEFI HTTP Boot: UEFI firmware natively supports HTTP/HTTPS boot
- Security: Boot file integrity verified via HTTPS chain of trust
Implementation on AWS
Certificate Provisioning:
- Use ACM certificate for public domain (free, auto-renewed)
- Use self-signed certificate for VPN-only access (add to iPXE trust store)
- Use ACM Private CA for internal PKI ($400/month - expensive for home lab)
ALB Configuration:
- HTTPS listener on port 443
- Target group pointing to EC2 boot server
- Security policy with TLS 1.2+ minimum
Alternative: Direct EC2 HTTPS:
- Run nginx/Apache with TLS on EC2 instance
- Access via VPN tunnel to private IP with HTTPS
- Simpler setup for VPN-only scenario
- Use Let’s Encrypt or self-signed certificate
Mutual TLS (mTLS) Support
AWS ALB supports mutual TLS authentication (as of 2022):
- Client Certificates: Require client certificates for authentication
- Trust Store: Upload trusted CA certificates to ALB
- Use Case: Ensure only authorized home lab servers can access boot files
- Integration: Combine with VPN for defense-in-depth
- Passthrough Mode: ALB can pass client cert to backend for validation
Routing and Load Balancing Capabilities
VPC Routing
- Route Tables: Define routes to direct traffic through VPN gateway
- Route Propagation: BGP route propagation for VPN connections
- Transit Gateway: Advanced multi-VPC/VPN routing (overkill for home lab)
Security Groups
- Stateful Firewall: Automatic return traffic handling
- Ingress/Egress Rules: Fine-grained control by protocol, port, source/destination
- Security Group Chaining: Reference security groups in rules (elegant for VPN setup)
- VPN Subnet Restriction: Allow traffic only from VPN-connected subnet
Network ACLs (Optional)
- Stateless Firewall: Subnet-level access control
- Defense in Depth: Additional layer beyond security groups
- Use Case: Probably unnecessary for simple VPN boot server
Cost Implications
Data Transfer Costs
- VPN Traffic: Data transfer through VPN gateway charged at standard rates
- Intra-Region: Free for traffic within same region/VPC
- Boot File Sizes: Typical kernel + initrd = 50-200MB per boot
- Monthly Estimate: 10 boots/month × 150MB = 1.5GB ≈ $0.14/month (US East egress)
Load Balancing Costs
- Application Load Balancer:
$0.0225/hour + $0.008 per LCU-hour ($16-20/month minimum) - Network Load Balancer:
$0.0225/hour + $0.006 per NLCU-hour ($16-18/month minimum) - For VPN Scenario: Load balancer unnecessary (single EC2 instance sufficient)
Compute Costs
- t3.micro Instance: ~$7.50/month (on-demand pricing, US East)
- t4g.micro Instance: ~$6.00/month (ARM-based, cheaper, sufficient for boot server)
- Reserved Instances: Up to 72% savings with 1-year or 3-year commitment
- Savings Plans: Flexible discounts for consistent compute usage
ACM Certificate Costs
- Public Certificates: Free when used with AWS services
- Private CA: $400/month (too expensive for home lab)
Comparison with Requirements
| Requirement | AWS Support | Implementation |
|---|
| TFTP | ⚠️ Via EC2, not ELB | Direct EC2 access via VPN |
| HTTP | ✅ Full support | EC2 or ALB |
| HTTPS | ✅ Full support | EC2 or ALB with ACM |
| VPN Integration | ✅ Native VPN | Site-to-Site VPN or self-managed |
| Load Balancing | ✅ ALB, NLB | Optional for HA |
| Certificate Mgmt | ✅ ACM (free) | Automatic renewal |
| Cost Efficiency | ✅ Low-cost instances | t4g.micro sufficient |
Recommendations
For VPN-Based Architecture (per ADR-0002)
EC2 Instance: Deploy single t4g.micro or t3.micro instance with:
- TFTP server (
tftpd-hpa or dnsmasq) - HTTP server (nginx or simple Python HTTP server)
- Optional HTTPS with Let’s Encrypt or self-signed certificate
VPN Connection: Connect home lab to AWS via:
- Site-to-Site VPN (IPsec) - managed service, higher cost (~$36/month)
- Self-managed WireGuard on EC2 - lower cost, more control
Security Groups: Restrict access to:
- UDP/69 (TFTP) from VPN security group only
- TCP/80 (HTTP) from VPN security group only
- TCP/443 (HTTPS) from VPN security group only
No Load Balancer: For home lab scale, direct EC2 access is sufficient
Health Monitoring: Use CloudWatch for instance and service health
If HA Required (Future Enhancement)
- Deploy multi-AZ EC2 instances with Network Load Balancer
- Use S3 as backend for boot files with EC2 serving as cache
- Implement auto-recovery with Auto Scaling Group (min=max=1)
References
2 - AWS WireGuard VPN Support
Analysis of WireGuard VPN deployment options on Amazon Web Services for secure site-to-site connectivity
WireGuard VPN Support on Amazon Web Services
This document analyzes options for deploying WireGuard VPN on AWS to establish secure site-to-site connectivity between the home lab and cloud-hosted network boot infrastructure.
WireGuard Overview
WireGuard is a modern VPN protocol that provides:
- Simplicity: Minimal codebase (~4,000 lines vs 100,000+ for IPsec)
- Performance: High throughput with low overhead
- Security: Modern cryptography (Curve25519, ChaCha20, Poly1305, BLAKE2s)
- Configuration: Simple key-based configuration
- Kernel Integration: Mainline Linux kernel support since 5.6
AWS Native VPN Support
Site-to-Site VPN (IPsec)
Status: ❌ WireGuard not natively supported
AWS’s managed Site-to-Site VPN supports:
- IPsec VPN: IKEv1, IKEv2 with pre-shared keys
- Redundancy: Two VPN tunnels per connection for high availability
- BGP Support: Dynamic routing via BGP
- Transit Gateway: Scalable multi-VPC VPN hub
Limitation: Site-to-Site VPN does not support WireGuard protocol natively.
Cost: Site-to-Site VPN
- VPN Connection: ~$0.05/hour = ~$36/month
- Data Transfer: Standard data transfer out rates (~$0.09/GB for first 10TB)
- Total Estimate: ~$36-50/month for managed IPsec VPN
Self-Managed WireGuard on EC2
Implementation Approach
Since AWS doesn’t offer managed WireGuard, deploy WireGuard on an EC2 instance:
Status: ✅ Fully supported via EC2
Architecture
graph LR
A[Home Lab] -->|WireGuard Tunnel| B[AWS EC2 Instance]
B -->|VPC Network| C[Boot Server EC2]
B -->|IP Forwarding| C
subgraph "Home Network"
A
D[UDM Pro]
D -.WireGuard Client.- A
end
subgraph "AWS VPC"
B[WireGuard Gateway EC2]
C[Boot Server EC2]
endEC2 Configuration
WireGuard Gateway Instance:
- Instance Type: t4g.micro or t3.micro ($6-7.50/month)
- OS: Ubuntu 22.04 LTS or Amazon Linux 2023 (native WireGuard support)
- Source/Dest Check: Disable to allow IP forwarding
- Elastic IP: Allocate Elastic IP for stable WireGuard endpoint
- Security Group: Allow UDP port 51820 from home lab public IP
Boot Server Instance:
- Network: Same VPC as WireGuard gateway
- Private IP Only: No Elastic IP (accessed via VPN)
- Route Traffic: Through WireGuard gateway instance
Installation Steps
# On EC2 Instance (Ubuntu 22.04+)
sudo apt update
sudo apt install wireguard wireguard-tools
# Generate server keys
wg genkey | tee /etc/wireguard/server_private.key | wg pubkey > /etc/wireguard/server_public.key
chmod 600 /etc/wireguard/server_private.key
# Configure WireGuard interface
sudo nano /etc/wireguard/wg0.conf
Example /etc/wireguard/wg0.conf on AWS EC2:
[Interface]
Address = 10.200.0.1/24
ListenPort = 51820
PrivateKey = <SERVER_PRIVATE_KEY>
PostUp = sysctl -w net.ipv4.ip_forward=1
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT
PostUp = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT
PostDown = iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
[Peer]
# Home Lab (UDM Pro)
PublicKey = <CLIENT_PUBLIC_KEY>
AllowedIPs = 10.200.0.2/32, 192.168.1.0/24
Corresponding config on UDM Pro:
[Interface]
Address = 10.200.0.2/24
PrivateKey = <CLIENT_PRIVATE_KEY>
[Peer]
PublicKey = <SERVER_PUBLIC_KEY>
Endpoint = <AWS_ELASTIC_IP>:51820
AllowedIPs = 10.200.0.0/24, 10.0.0.0/16
PersistentKeepalive = 25
Enable and Start WireGuard
# Enable IP forwarding permanently
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
# Enable WireGuard interface
sudo systemctl enable wg-quick@wg0
sudo systemctl start wg-quick@wg0
# Verify status
sudo wg show
AWS VPC Configuration
Security Groups
Create security group for WireGuard gateway:
aws ec2 create-security-group \
--group-name wireguard-gateway-sg \
--description "WireGuard VPN gateway" \
--vpc-id vpc-xxxxxx
aws ec2 authorize-security-group-ingress \
--group-id sg-xxxxxx \
--protocol udp \
--port 51820 \
--cidr <HOME_LAB_PUBLIC_IP>/32
Allow SSH for management (optional, restrict to trusted IP):
aws ec2 authorize-security-group-ingress \
--group-id sg-xxxxxx \
--protocol tcp \
--port 22 \
--cidr <TRUSTED_IP>/32
Disable Source/Destination Check
Required for IP forwarding to work:
aws ec2 modify-instance-attribute \
--instance-id i-xxxxxx \
--no-source-dest-check
Elastic IP Allocation
Allocate and associate Elastic IP for stable endpoint:
aws ec2 allocate-address --domain vpc
aws ec2 associate-address \
--instance-id i-xxxxxx \
--allocation-id eipalloc-xxxxxx
Cost: Elastic IP is free when associated with running instance, but charged ~$3.60/month if unattached.
Route Table Configuration
Add route to direct home lab subnet traffic through WireGuard gateway:
aws ec2 create-route \
--route-table-id rtb-xxxxxx \
--destination-cidr-block 192.168.1.0/24 \
--instance-id i-xxxxxx
This routes home lab subnet (192.168.1.0/24) through the WireGuard gateway instance.
UDM Pro WireGuard Integration
Native Support
Status: ✅ WireGuard supported natively (UniFi OS 1.12.22+)
The UniFi Dream Machine Pro includes native WireGuard VPN support:
- GUI Configuration: Web UI for WireGuard VPN setup
- Site-to-Site: Support for site-to-site VPN tunnels
- Performance: Hardware acceleration for encryption (if available)
- Routing: Automatic route injection for remote subnets
Configuration Steps on UDM Pro
Network Settings → VPN:
- Create new VPN connection
- Select “WireGuard”
- Generate key pair or import existing
Peer Configuration:
- Peer Public Key: AWS EC2 WireGuard instance’s public key
- Endpoint: AWS Elastic IP address
- Port: 51820
- Allowed IPs: AWS VPC CIDR (e.g., 10.0.0.0/16)
- Persistent Keepalive: 25 seconds
Route Injection:
- UDM Pro automatically adds routes to AWS subnets
- Home lab servers can reach AWS boot server via VPN
Firewall Rules:
- Add firewall rule to allow boot traffic (TFTP, HTTP) from LAN to VPN
Alternative: Manual WireGuard on UDM Pro
If native support is insufficient, use wireguard-go via udm-utilities:
- Repository: boostchicken/udm-utilities
- Script:
on_boot.d script to start WireGuard on boot - Persistence: Survives firmware updates with on-boot script
Throughput
WireGuard on EC2 performance varies by instance type:
- t4g.micro (2 vCPU, ARM): ~100-300 Mbps
- t3.micro (2 vCPU, x86): ~100-300 Mbps
- t3.small (2 vCPU): ~500-800 Mbps
- t3.medium (2 vCPU): ~1+ Gbps
For network boot (typical boot = 50-200MB), even t4g.micro is sufficient:
- Boot Time: 150MB at 100 Mbps = ~12 seconds transfer time
- Recommendation: t4g.micro adequate and most cost-effective
Latency
- VPN Overhead: WireGuard adds minimal latency (~1-5ms)
- AWS Network: Low-latency network infrastructure
- Total Latency: Primarily dependent on home ISP and AWS region proximity
CPU Usage
- Encryption: ChaCha20 is CPU-efficient
- Kernel Module: Minimal CPU overhead in kernel space
- t4g.micro: Sufficient CPU for home lab VPN throughput
- ARM Advantage: t4g instances use Graviton processors (better price/performance)
Security Considerations
Key Management
- Private Keys: Store securely, never commit to version control
- Key Rotation: Rotate keys periodically (e.g., annually)
- Secrets Manager: Store WireGuard private keys in AWS Secrets Manager
- Retrieve at instance startup via user data script
- Avoid storing in AMIs or instance metadata
- IAM Role: Grant EC2 instance IAM role to read secret
Firewall Hardening
- Security Group Restriction: Limit WireGuard port to home lab public IP only
- Least Privilege: Boot server security group allows only VPN security group
- No Public Access: Boot server has no Elastic IP or public route
Monitoring and Alerts
- CloudWatch Logs: Stream WireGuard logs to CloudWatch
- CloudWatch Alarms: Alert on VPN tunnel down (no recent handshakes)
- VPC Flow Logs: Monitor VPN traffic patterns
DDoS Protection
- UDP Amplification: WireGuard resistant to DDoS amplification attacks
- AWS Shield: Basic DDoS protection included free on all AWS resources
- Shield Advanced: Optional ($3,000/month - overkill for VPN endpoint)
High Availability Options
Multi-AZ Failover
Deploy WireGuard gateways in multiple Availability Zones:
- Primary: us-east-1a WireGuard instance
- Secondary: us-east-1b WireGuard instance
- Failover: UDM Pro switches endpoints if primary fails
- Cost: Doubles instance costs (~$12-15/month for 2 instances)
Auto Scaling Group (Single Instance)
Use Auto Scaling Group with min=max=1 for auto-recovery:
- Health Checks: EC2 status checks
- Auto-Recovery: ASG replaces failed instance automatically
- Elastic IP: Reassociate Elastic IP to new instance via Lambda/script
- Limitation: Brief downtime during recovery (~2-5 minutes)
Health Monitoring
Monitor WireGuard tunnel health with CloudWatch custom metrics:
# On EC2 instance, run periodically via cron
#!/bin/bash
HANDSHAKE=$(wg show wg0 latest-handshakes | awk '{print $2}')
NOW=$(date +%s)
AGE=$((NOW - HANDSHAKE))
aws cloudwatch put-metric-data \
--namespace WireGuard \
--metric-name TunnelAge \
--value $AGE \
--unit Seconds
Alert if handshake age exceeds threshold (e.g., 180 seconds).
User Data Script for Auto-Configuration
EC2 user data script to configure WireGuard on launch:
#!/bin/bash
# Install WireGuard
apt update && apt install -y wireguard wireguard-tools
# Retrieve private key from Secrets Manager
aws secretsmanager get-secret-value \
--secret-id wireguard-server-key \
--query SecretString \
--output text > /etc/wireguard/server_private.key
chmod 600 /etc/wireguard/server_private.key
# Configure interface (full config omitted for brevity)
# ...
# Enable and start WireGuard
systemctl enable wg-quick@wg0
systemctl start wg-quick@wg0
Requires IAM instance role with secretsmanager:GetSecretValue permission.
Cost Analysis
Self-Managed WireGuard on EC2
| Component | Cost (US East) |
|---|
| t4g.micro instance (730 hrs/month) | ~$6.00 |
| Elastic IP (attached) | $0.00 |
| Data transfer out (1GB/month) | ~$0.09 |
| Monthly Total | ~$6.09 |
| Annual Total | ~$73 |
With Reserved Instance (1-year, no upfront):
| Component | Cost |
|---|
| t4g.micro RI (1-year) | ~$3.50/month |
| Elastic IP | $0.00 |
| Data transfer | ~$0.09 |
| Monthly Total | ~$3.59 |
| Annual Total | ~$43 |
Site-to-Site VPN (IPsec - if WireGuard not used)
| Component | Cost |
|---|
| VPN Connection (2 tunnels) | ~$36 |
| Data transfer (1GB/month) | ~$0.09 |
| Monthly Total | ~$36 |
| Annual Total | ~$432 |
Cost Savings: Self-managed WireGuard saves ~$360/year vs Site-to-Site VPN (or ~$390/year with Reserved Instance).
Comparison with Requirements
| Requirement | AWS Support | Implementation |
|---|
| WireGuard Protocol | ✅ Via EC2 | Self-managed on instance |
| Site-to-Site VPN | ✅ Yes | WireGuard tunnel |
| UDM Pro Integration | ✅ Native support | WireGuard peer config |
| Cost Efficiency | ✅ Very low cost | t4g.micro ~$6/month (on-demand) |
| Performance | ✅ Sufficient | 100+ Mbps on t4g.micro |
| Security | ✅ Modern crypto | ChaCha20, Curve25519 |
| HA (optional) | ⚠️ Manual setup | Multi-AZ or ASG |
Recommendations
For Home Lab VPN (per ADR-0002)
Self-Managed WireGuard: Deploy on EC2 t4g.micro instance
- Cost: ~$6/month on-demand, ~$3.50/month with Reserved Instance
- Performance: Sufficient for network boot traffic
- Simplicity: Easy to configure and maintain
Single AZ Deployment: Unless HA required, single instance adequate
- Region Selection: Choose region closest to home lab for lowest latency
- AZ: Single AZ sufficient (boot server not mission-critical)
UDM Pro Native WireGuard: Use built-in WireGuard client
- Configuration: Add AWS instance as WireGuard peer in UDM Pro UI
- Route Injection: UDM Pro automatically routes AWS subnets
Security Best Practices:
- Store WireGuard private key in Secrets Manager
- Restrict security group to home lab public IP only
- Use user data script to retrieve key and configure on boot
- Enable CloudWatch logging for VPN events
- Assign IAM instance role with minimal permissions
Monitoring: Set up CloudWatch alarms for:
- Instance status check failures
- High CPU usage
- VPN tunnel age (custom metric)
Cost Optimization
- Reserved Instance: Commit to 1-year Reserved Instance for ~40% savings
- Spot Instance: Consider Spot for even lower cost (~70% savings), but adds complexity (handle interruptions)
- ARM Architecture: Use t4g (Graviton) for 20% better price/performance vs t3
Future Enhancements
- HA Setup: Deploy secondary WireGuard instance in different AZ
- Automated Failover: Lambda function to reassociate Elastic IP on failure
- IPv6 Support: Enable WireGuard over IPv6 if home ISP supports
- Mesh VPN: Expand to mesh topology if multiple sites added
References