oxc-project__oxc/ARCHITECTURE.md

# Architecture

## System Overview

Oxc (The Oxidation Compiler) is a collection of high-performance JavaScript and TypeScript tools written in Rust. The system is designed as a modular, composable set of compiler components that can be used independently or together to build complete toolchains for JavaScript/TypeScript development.

### Core Mission

- **Performance**: Deliver faster performance than existing JavaScript tools
- **Correctness**: Maintain compatibility with JavaScript/TypeScript standards
- **Modularity**: Enable users to compose tools according to their specific needs
- **Developer Experience**: Provide excellent error messages and tooling integration

## High-Level Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                          Applications                           │
├─────────────────────────────────────────────────────────────────┤
│  oxlint  │  Language Server  │  NAPI Bindings  │  Future Tools  │
├─────────────────────────────────────────────────────────────────┤
│                        Core Libraries                           │
├─────────────────────────────────────────────────────────────────┤
│ Parser │ Semantic │ Linter │ Transformer │ Minifier │ Codegen   │
├─────────────────────────────────────────────────────────────────┤
│                    Foundation Libraries                         │
├─────────────────────────────────────────────────────────────────┤
│    AST    │  Allocator  │  Diagnostics  │   Span   │  Syntax    │
└─────────────────────────────────────────────────────────────────┘
```

## Architecture Principles

### 1. Zero-Copy Architecture

The system is built around an arena allocator (`oxc_allocator`) that enables zero-copy operations throughout the compilation pipeline. All AST nodes are allocated in a single arena, eliminating the need for reference counting or garbage collection.

### 2. Visitor Pattern

AST traversal is implemented using the visitor pattern (`oxc_ast_visit`) with automatic visitor generation through procedural macros. This ensures type safety and performance while maintaining code clarity.

### 3. Shared Infrastructure

Common functionality like error reporting (`oxc_diagnostics`), source positions (`oxc_span`), and syntax definitions (`oxc_syntax`) are shared across all components to ensure consistency.

## Core Components

### Foundation Layer

#### oxc_allocator

- **Purpose**: Arena-based memory allocator for zero-copy operations
- **Key Features**:
  - Single allocation arena for entire compilation unit
  - Eliminates need for Rc/Arc in hot paths
  - Enables structural sharing of AST nodes
- **Dependencies**: None (foundational)

#### oxc_span

- **Purpose**: Source position tracking and text manipulation
- **Key Features**:
  - Byte-based indexing for UTF-8 correctness
  - Efficient span operations for source maps
  - Integration with diagnostic reporting
- **Dependencies**: None (foundational)

#### oxc_syntax

- **Purpose**: JavaScript/TypeScript language definitions
- **Key Features**:
  - Token definitions and keyword mappings
  - Language feature flags and compatibility
  - Shared syntax validation logic
- **Dependencies**: oxc_span

#### oxc_diagnostics

- **Purpose**: Error reporting and diagnostic infrastructure
- **Key Features**:
  - Rich error messages with source context
  - Multiple output formats (JSON, pretty-printed)
  - Integration with language server protocol
- **Dependencies**: oxc_span

#### oxc_ast

- **Purpose**: Abstract Syntax Tree definitions and utilities
- **Key Features**:
  - Complete JavaScript/TypeScript AST coverage
  - Generated visitor traits for type safety
  - Serialization support for caching
- **Dependencies**: oxc_allocator, oxc_span, oxc_syntax

##### AST Design Principles

The Oxc AST differs significantly from the [estree](https://github.com/estree/estree) AST specification by removing ambiguous nodes and introducing distinct types. While many existing JavaScript tools rely on estree as their AST specification, a notable drawback is its abundance of ambiguous nodes that often leads to confusion during development.

For example, instead of using a generic estree `Identifier`, the Oxc AST provides specific types such as:

- `BindingIdentifier` - for variable declarations and bindings
- `IdentifierReference` - for variable references
- `IdentifierName` - for property names and labels

This clear distinction greatly enhances the development experience by aligning more closely with the ECMAScript specification and providing better type safety.

### Core Processing Layer

#### oxc_parser

- **Purpose**: JavaScript/TypeScript parsing
- **Key Features**:
  - Hand-written recursive descent parser
  - Full ES2024+ and TypeScript support
  - Preservation of comments and trivia
- **Dependencies**: oxc_allocator, oxc_ast, oxc_diagnostics, oxc_span, oxc_syntax

#### oxc_semantic

- **Purpose**: Semantic analysis and symbol resolution
- **Key Features**:
  - Scope chain construction
  - Symbol table generation
  - Dead code detection
- **Dependencies**: oxc_ast, oxc_cfg, oxc_diagnostics, oxc_span, oxc_syntax

#### oxc_linter

- **Purpose**: ESLint-compatible linting engine
- **Key Features**:
  - 200+ built-in rules
  - Plugin architecture for custom rules
  - Automatic fixing for many rules
  - Configuration compatibility with ESLint
- **Dependencies**: oxc_ast, oxc_semantic, oxc_diagnostics, oxc_cfg

#### oxc_transformer

- **Purpose**: Code transformation and transpilation
- **Key Features**:
  - TypeScript to JavaScript transformation
  - Modern JavaScript feature transpilation
  - React JSX transformation
  - Babel plugin compatibility layer
- **Dependencies**: oxc_ast, oxc_semantic, oxc_allocator

#### oxc_minifier

- **Purpose**: Code size optimization
- **Key Features**:
  - Dead code elimination
  - Constant folding and propagation
  - Identifier mangling integration
  - Statement and expression optimization
- **Dependencies**: oxc_ast, oxc_semantic, oxc_mangler

#### oxc_codegen

- **Purpose**: AST to source code generation
- **Key Features**:
  - Configurable output formatting
  - Source map generation
  - Comment preservation options
  - Minified and pretty-printed output modes
- **Dependencies**: oxc_ast, oxc_span

### Application Layer

#### oxlint (apps/oxlint)

- **Purpose**: Command-line linter application
- **Key Features**:
  - File discovery and parallel processing
  - Configuration file support
  - Multiple output formats
  - Integration with CI/CD systems
- **Dependencies**: oxc_linter, oxc_parser, oxc_semantic

#### Language Server (oxc_language_server)

- **Purpose**: LSP implementation for editor integration
- **Key Features**:
  - Real-time diagnostics
  - Go-to-definition and references
  - Symbol search and completion
- **Dependencies**: All core components

#### NAPI Bindings (napi/\*)

- **Purpose**: Node.js integration layer
- **Key Features**:
  - Parser bindings for JavaScript tooling
  - Linter integration for build tools
  - Transform pipeline for bundlers
  - Async processing support
- **Dependencies**: Core components + Node.js FFI

## Data Flow

### Compilation Pipeline

1. **Input**: Source text + configuration
2. **Lexing/Parsing**: `oxc_parser` → AST + comments
3. **Semantic Analysis**: `oxc_semantic` → Symbol table + scope info
4. **Processing**: Tool-specific analysis (linting, transformation, etc.)
5. **Output**: Results (diagnostics, transformed code, etc.)

### Memory Management Flow

```
Source Text → Arena Allocator → AST Nodes → Visitors → Results
     ↓              ↓              ↓           ↓          ↓
   UTF-8          Arena         Borrowed    Zero-copy   Owned
  String         Memory         References  Processing  Output
```

## Quality Attributes

### Performance

- **Target**: 10-100x faster than comparable tools
- **Strategies**:
  - Arena allocation for memory efficiency
  - Zero-copy data structures
  - Parallel processing where possible
  - Minimal allocations in hot paths

#### Parser Performance Implementation

- AST is allocated in a memory arena (oxc_allocator) for fast AST memory allocation and deallocation
- Short strings are inlined by [CompactString](https://crates.io/crates/compact_str)
- No other heap allocations are done except the above two
- Scope binding, symbol resolution and some syntax errors are not done in the parser, they are delegated to the semantic analyzer

#### Linter Performance Implementation

- Oxc parser is used for optimal performance
- AST visit is a fast operation due to linear memory scan from the memory arena
- Files are linted in a multi-threaded environment, so scales with the total number of CPU cores
- Every single lint rule is tuned for performance

### Correctness

- **Target**: 100% compatibility with language standards
- **Strategies**:
  - Comprehensive test suites
  - Real-world codebase testing
  - Conformance testing against official specs
  - Conservative error handling

### Maintainability

- **Target**: Clear, reviewable, extensible codebase
- **Strategies**:
  - Strong type system usage
  - Procedural macro code generation
  - Clear separation of concerns
  - Comprehensive documentation

### Usability

- **Target**: Drop-in replacement for existing tools
- **Strategies**:
  - Configuration compatibility
  - Familiar CLI interfaces
  - Rich error messages
  - Editor integration

## Technical Constraints

### Language Choice

- **Rust**: Chosen for memory safety, performance, and zero-cost abstractions
- **MSRV**: N-2 policy for stability

### Memory Model

- **Arena Allocation**: Single arena per compilation unit
- **Lifetime Management**: Explicit lifetimes tied to arena
- **No Garbage Collection**: Manual memory management for predictable performance

### Threading Model

- **File-level Parallelism**: Multiple files processed in parallel
- **Single-threaded Pipeline**: Each file processed by single thread
- **Shared State**: Minimal shared state to avoid synchronization overhead

### Compatibility Requirements

- **JavaScript**: ES2024+ compatibility
- **TypeScript**: Latest TypeScript syntax support
- **Node.js**: LTS versions through NAPI bindings
- **Editors**: LSP compatibility for all major editors

## Design Decisions

### Arena Allocator Choice

**Decision**: Use custom arena allocator instead of Rc/Arc
**Rationale**:

- Eliminates reference counting overhead
- Enables zero-copy string operations
- Simplifies memory management
- Improves cache locality

**Trade-offs**:

- ✅ 10-50% performance improvement
- ✅ Simplified ownership model
- ❌ Requires lifetime management
- ❌ Less flexible memory patterns

### Hand-written Parser

**Decision**: Implement recursive descent parser instead of parser generator
**Rationale**:

- Easier debugging and maintenance
- More efficient generated code
- Faster compilation times

**Trade-offs**:

- ✅ Better performance and error messages
- ✅ More maintainable code
- ❌ More manual implementation work
- ❌ Higher risk of parser bugs

### Visitor Pattern

**Decision**: Use visitor pattern with procedural macros
**Rationale**:

- Type-safe AST traversal
- Automatic visitor generation
- Consistent patterns across tools
- Efficient dispatch

**Trade-offs**:

- ✅ Type safety and performance
- ✅ Reduced boilerplate code
- ❌ Compile-time complexity
- ❌ Learning curve for contributors

## Future Considerations

### Planned Extensions

- **Formatter**: Complete code formatting tool
- **Bundler**: Integration with bundling workflows
- **Type Checker**: Full TypeScript type checking
- **Plugin System**: User-defined transformations

### Scalability Concerns

- **Large Codebases**: Processing optimization improvements
- **Memory Usage**: Streaming processing for huge files
- **Parallel Processing**: Fine-grained parallelization

### Technology Evolution

- **Rust Evolution**: Leveraging new language features
- **JavaScript Standards**: Keeping pace with TC39 proposals
- **Editor Integration**: Advanced IDE features

## Development Infrastructure

### Test Infrastructure

Correctness and reliability are taken extremely seriously in Oxc. We spend significant effort on strengthening the test infrastructure to prevent problems from propagating to downstream tools:

- **Conformance Testing**: Test262, Babel, and TypeScript conformance suites
- **Fuzzing**: Extensive fuzzing to discover edge cases
- **Snapshot Testing**: Linter diagnostic snapshots for regression prevention
- **Ecosystem CI**: Testing against real-world codebases
- **Idempotency Testing**: Ensuring transformations are stable
- **Code Coverage**: Comprehensive coverage tracking
- **End-to-End Testing**: Testing against top 3000 npm packages

### Build and Development Tools

- **Rust**: MSRV 1.86.0+ with clippy and rustfmt integration
- **Just**: Command runner for development tasks (`just --list` for available commands)
- **Performance Monitoring**: Continuous benchmarking and performance regression detection
- **Cross-platform**: Support for Linux, macOS, and Windows
- **CI/CD**: Automated testing, building, and publishing pipelines

For detailed development guidelines, see [CONTRIBUTING.md](./CONTRIBUTING.md) and [AGENTS.md](./AGENTS.md).

---

This architecture document follows the [architecture.md](https://architecture.md/) format for documenting software architecture decisions and system design.