OrthoRoute/docs/ORP_ORS_file_formats.md

# OrthoRoute File Formats

This document describes the file formats used by OrthoRoute for cloud routing workflows.

## Overview

OrthoRoute uses two specialized file formats to enable routing PCBs without KiCad:

- **`.ORP`** (OrthoRoute PCB) - Contains board geometry and design rules
- **`.ORS`** (OrthoRoute Solution) - Contains routing results (traces and vias)

Both formats use **JSON with gzip compression** for efficient storage and transfer.

---

## .ORP Format (OrthoRoute PCB)

**Purpose:** Export board data from KiCad for routing on remote machines

**Creation:** File → Export PCB (Ctrl+E) in OrthoRoute GUI

**Contents:**
- Board metadata (filename, dimensions, layer count)
- Pad locations and net assignments
- Net definitions (terminal positions)
- Design rules (clearances, track widths, via sizes)
- Grid parameters

### File Structure

```json
{
  "format_version": "1.0",

  "board_metadata": {
    "filename": "MyBoard.kicad_pcb",
    "bounds": {
      "x_min": 100.0,
      "y_min": 50.0,
      "x_max": 200.0,
      "y_max": 150.0
    },
    "layer_count": 6
  },

  "pads": [
    {
      "position": [150.5, 100.2],
      "net": "VCC",
      "drill": 0.8,
      "layer_mask": 4227858432
    }
  ],

  "nets": {
    "VCC": [[150.5, 100.2], [175.3, 120.5]],
    "GND": [[160.0, 110.0], [180.0, 130.0]]
  },

  "drc_rules": {
    "clearance": 0.15,
    "track_width": 0.25,
    "via_diameter": 0.6,
    "via_drill": 0.3,
    "min_drill": 0.2
  },

  "grid_parameters": {
    "pitch": 0.4
  }
}
```

### Coordinate System

- **Units:** Millimeters (mm)
- **Origin:** Board lower-left corner (x_min, y_min)
- **Axes:** X increases right, Y increases up
- **Layer IDs:** KiCad layer numbering (0=F.Cu, 31=B.Cu, 1-30=Internal)

### Typical File Sizes

- Small board (50 nets): ~15 KB compressed
- Medium board (200 nets): ~40 KB compressed
- Large board (500 nets): ~80 KB compressed

---

## .ORS Format (OrthoRoute Solution)

**Purpose:** Store routing results for import back into KiCad

**Creation:** Automatically generated by `python main.py headless <board>.ORP`

**Contents:**
- Routed traces (per net, per layer)
- Via locations and layer spans
- Iteration metrics (convergence history)
- Final routing statistics

### File Structure

```json
{
  "format_version": "1.0",

  "geometry": {
    "by_net": {
      "VCC": {
        "net_id": "VCC",
        "tracks": [
          {
            "layer": "In1.Cu",
            "start": {"x": 150.5, "y": 100.2},
            "end": {"x": 151.5, "y": 100.2},
            "width": 0.25
          }
        ],
        "vias": [
          {
            "position": {"x": 150.5, "y": 100.2},
            "from_layer": "F.Cu",
            "to_layer": "In1.Cu",
            "diameter": 0.6,
            "drill": 0.3
          }
        ]
      }
    },
    "all_tracks": [...],
    "all_vias": [...],
    "layer_usage": {
      "In1.Cu": 250,
      "In2.Cu": 180
    }
  },

  "iteration_metrics": [
    {
      "iteration": 1,
      "overuse_count": 42000,
      "nets_routed": 512,
      "overflow_cost": 85000.0,
      "wirelength": 15000.5,
      "via_count": 1250,
      "iteration_time_seconds": 45.2
    }
  ],

  "metadata": {
    "export_timestamp": "2025-11-13T18:19:50Z",
    "orthoroute_version": "0.1.0",
    "board_name": "MyBoard.kicad_pcb",
    "total_iterations": 71,
    "converged": true,
    "total_time_seconds": 1680.5,
    "notes": ""
  },

  "statistics": {
    "total_tracks": 4128,
    "total_vias": 2593,
    "total_wirelength_mm": 12500.8,
    "nets_routed": 512,
    "final_overuse_count": 0,
    "final_overflow_cost": 0.0,
    "converged": true,
    "iterations_completed": 71
  }
}
```

### Coordinate System

Same as .ORP format - all coordinates in millimeters relative to board origin.

### Layer Naming

Layers use KiCad naming convention:
- `F.Cu` - Front copper
- `B.Cu` - Back copper
- `In1.Cu` through `In30.Cu` - Internal layers

### Typical File Sizes

- Small board (50 nets, 500 tracks): ~80 KB compressed
- Medium board (200 nets, 2000 tracks): ~250 KB compressed
- Large board (500 nets, 4000 tracks): ~450 KB compressed

---

## Workflow

```
┌─────────────┐
│   KiCad     │
│   Design    │
└──────┬──────┘
       │
       │ Export (Ctrl+E)
       ▼
┌─────────────┐
│  .ORP File  │  ◄── Upload to cloud GPU
└──────┬──────┘
       │
       │ python main.py headless board.ORP
       ▼
┌─────────────┐
│  .ORS File  │  ◄── Download from cloud
└──────┬──────┘
       │
       │ Import (Ctrl+I)
       ▼
┌─────────────┐
│   KiCad     │
│  (Routed)   │
└─────────────┘
```

---

## File Format Versioning

**Current Version:** 1.0

Both .ORP and .ORS files include a `"format_version"` field. Future versions of OrthoRoute will maintain backward compatibility by checking this version and handling older formats appropriately.

### Version History

- **1.0** (2025-11-13) - Initial format specification
  - Basic board geometry export
  - Net-organized routing results
  - Iteration metrics and statistics

---

## Working with Files

### Reading Files Manually

Both formats are gzip-compressed JSON, so you can inspect them with:

```python
import gzip
import json

# Read ORP file
with gzip.open('board.ORP', 'rt', encoding='utf-8') as f:
    orp_data = json.load(f)
    print(f"Board: {orp_data['board_metadata']['filename']}")
    print(f"Nets: {len(orp_data['nets'])}")

# Read ORS file
with gzip.open('board.ORS', 'rt', encoding='utf-8') as f:
    ors_data = json.load(f)
    print(f"Tracks: {ors_data['statistics']['total_tracks']}")
    print(f"Vias: {ors_data['statistics']['total_vias']}")
    print(f"Converged: {ors_data['metadata']['converged']}")
```

### Programmatic Access

Use the OrthoRoute serialization module:

```python
from orthoroute.infrastructure.serialization import (
    import_board_from_orp,
    export_board_to_orp,
    import_solution_from_ors,
    export_solution_to_ors
)

# Import board
board_data = import_board_from_orp('board.ORP')

# Import solution
geometry_data, metadata = import_solution_from_ors('board.ORS')

# Access data
print(f"Total nets: {len(geometry_data['by_net'])}")
print(f"Convergence: {metadata['metadata']['converged']}")
print(f"Total time: {metadata['metadata']['total_time_seconds']:.1f}s")
```

---

## Cloud Provider Setup

The .ORP/.ORS workflow is designed for cloud GPU routing. Typical setup:

1. **Export** .ORP file locally
2. **Upload** to cloud instance via SCP:
   ```bash
   scp board.ORP user@gpu-instance:/workspace/
   ```
3. **Route** on cloud:
   ```bash
   ssh user@gpu-instance
   cd /workspace
   python main.py headless board.ORP
   ```
4. **Download** .ORS file:
   ```bash
   scp user@gpu-instance:/workspace/board.ORS ./
   ```
5. **Import** in KiCad with Ctrl+I

**Recommended Providers:**
- [Vast.ai](https://vast.ai) - ~$0.37/hr for RTX 5090
- [RunPod](https://runpod.io) - ~$0.40/hr for RTX 4090
- [Lambda Labs](https://lambdalabs.com) - ~$1.10/hr for A100

---

## File Validation

OrthoRoute validates files on import:

**ORP Validation:**
- ✓ Format version is supported
- ✓ Required fields present (board_metadata, pads, nets, drc_rules)
- ✓ Coordinate ranges are reasonable
- ✓ Net references are consistent

**ORS Validation:**
- ✓ Format version is supported
- ✓ Required sections present (geometry, metadata, statistics)
- ✓ Track coordinates are valid
- ✓ Layer names are recognized

Invalid files will show a clear error message indicating what's wrong.

---

## Limitations and Future Work

**Current Limitations:**

1. **Existing traces not exported** - .ORP assumes a clean board
2. **Keepout zones not serialized** - Must manually avoid keepouts
3. **No parameter override** - Uses default routing parameters
4. **No component information** - Only pads and nets exported

**Planned Enhancements:**

- [ ] Export existing traces for partial routing
- [ ] Include keepout zone definitions
- [ ] Support custom parameter JSON files
- [ ] Component placement data for reference
- [ ] Differential pair information
- [ ] Impedance control specifications

---

## Technical Notes

### Why gzip compression?

Plain JSON files can be 1-2 MB for large boards. Gzip compression reduces this to ~50-100 KB (95%+ reduction), making uploads/downloads much faster.

### Why JSON instead of binary?

JSON is:
- Human-readable and debuggable
- Easy to parse in any language
- Self-documenting
- Extensible for future versions
- Well-supported by all platforms

The gzip compression eliminates most of the size overhead of JSON.

### Thread safety

Both import and export functions are thread-safe and can be called concurrently from multiple threads if needed.

---

**Last Updated:** November 13, 2025
**Format Version:** 1.0
**OrthoRoute Version:** 0.1.0