In-Depth Exploration of Migrating TypeScript to Go: Decisions, Advantages, and Future Prospects

I. Project Background and Origin

(I) The Origin of the Project Code Name

The code name of the new TypeScript migration project is Corsa. The old codebase, Strata, was once the initial code name of TypeScript, which began in the internal development stage at the end of 2010 or the beginning of 2011. The initial team consisted of Steve Lucco, Anders Hejlsberg, and Luke. Steve wrote the original prototype compiler, extracting and modifying the scanner and parser from the JavaScript engine of Internet Explorer. It was a C# codebase used for the proof of concept.

(II) Performance Issues Driving the Change

In the ECMAScript community, it is a trend to migrate highly tool-dependent projects to native code, such as esbuild and swc. TypeScript faces performance and scalability issues. As the project keeps growing, the compiler exerts more pressure on the V8 and JavaScript engines. The startup time becomes longer due to the addition of new features. Previous optimizations could only bring a 5% - 10% improvement, reaching the limit of performance optimization.

II. Reasons for Choosing the Go Language

(I) Comparison with the Rust Language

Memory Management and Compatibility: The existing TypeScript codebase assumes the existence of automatic garbage collection. However, the memory management in Rust is not automatic. Its borrow checker has strict constraints on the ownership of data structures and prohibits cyclic data structures. TypeScript's data structures, such as the Abstract Syntax Tree (AST), make extensive use of circular references. Migrating to Rust requires redesigning the data structures, which adds great difficulty. Therefore, Rust is basically ruled out.
Developer Experience and Learning Cost: It is easier to transition from JavaScript to Go than to Rust. In some aspects, Go code is similar to JavaScript code. When dealing with complex or recursive structures in Rust, it is more difficult to understand the evolution process from TypeScript code. From the perspective of human resources, choosing Go has more advantages.

(II) Comparison with the C# Language

Language Design Orientation: Go is a language that gives more priority to native code. It has the function of automatic garbage collection and is more expressive in terms of data structure layout and inline structures. C# is somewhat bytecode-oriented. Although it has ahead-of-time compilation, it is not available on all platforms, and it was not designed with native performance optimization as the goal from the beginning.
Differences in Programming Paradigms: The JavaScript codebase of TypeScript adopts a highly functional programming style, and the core compiler rarely uses classes. Go also focuses on functions and data structures. In contrast, C# is mainly object-oriented programming (OOP). Migrating to C# requires switching the programming paradigm, which increases the migration friction.

(III) The Advantageous Fit of the Go Language

The Go language can provide excellently optimized native code on all mainstream platforms. It has good expressive ability for data structures, allowing cyclic data structures and inline data structures. It has the capabilities of automatic garbage collection and concurrent access to shared memory, as well as a good toolchain and excellent support from VS Code and other tools. It meets the multi-faceted needs of the TypeScript migration and stands out among many languages.

III. Challenges Faced by the Project and Solutions

(I) The Trade-off of Giving Up Bootstrapping

A bootstrapping language is a language written in itself. TypeScript was previously a bootstrapping language. There are concerns about giving up bootstrapping after migrating to Go, but for a 10-fold performance improvement, the team still chooses to migrate. However, some parts written in JavaScript will be retained, such as the language service part. The team is exploring solutions for building an API between the native part (Go) and consumers of other languages.

(II) Efforts to Ensure Compatibility

TypeScript does not have an official specification, and the reference implementation is similar to a specification. When migrating to Go, it is necessary to maintain semantic consistency. The team's goal is 99.99% compatibility and hopes to produce exactly the same errors for the same codebase. Currently, the open-source compiler can compile and check all of Visual Studio Code, and it can run 20,000 conformance tests without crashing. The team is analyzing the error baseline and eliminating differences, aiming to become a plug-and-play replacement for the old compiler.

(III) The Problem of Determinacy in Type Sorting

The old compiler used a simple non-deterministic type sorting method, which was deterministic in a single-threaded environment but non-deterministic in a multi-threaded concurrent environment. The new codebase needs to introduce deterministic type sorting, which leads to different type orders from the old compiler in some cases. Especially, the order of union types is important in some scenarios, and the team is working on solving these problems.

(IV) The Dilemma of API Design

Almost all of the internal structure of the compiler in the old codebase was exposed as an API. The new codebase needs to redesign the API and consider ensuring the efficiency of the API during inter-process communication. Currently, the team is exploring how to provide a versionable and modern API for the new codebase.

IV. The Application and Advantages of Concurrency in the Project

(I) The Functional Programming Foundation of the Compiler Facilitates Concurrency

The TypeScript compiler originally adopted a functional programming model and made extensive use of immutability to ensure safe sharing. For example, the AST after scanning, parsing, and binding is basically considered immutable. Multiple type checkers can process the same AST simultaneously, which provides a good foundation for concurrent processing, even though JavaScript itself does not have a concurrent mechanism for shared memory.

(II) The Implementation of Concurrency in the Parsing Stage

The parsing task is very suitable for parallelization. The parsing work of each source file can be completely independently completed. For example, if there are 5000 source files and 8 CPUs, the files can be divided into eight parts, and each CPU processes a part. In the shared memory space, after completion, the part that builds and links all the data structures is carried out. It is very simple to implement the concurrency of the parsing stage in Go. It probably only requires about 10 lines of code to run the operation in a goroutine, and at the same time, use a mutex to protect the shared resources, which can improve the performance by 3 to 4 times.

(III) The Concurrency Scheme for the Type Checking Stage

Since the type checker requires a global view of the program, it cannot be completely independent like the parsing process. The team divides the program into several parts (currently hard-coded as four) and creates four type checkers. Each checker checks the assigned part of the files. They share the underlying immutable AST and build their own type states. Although this method will consume about 20% more memory (due to type duplication), it can achieve an additional performance improvement of about 3 times. Combined with the 3-fold performance improvement brought by the native code, the overall performance improvement can reach 10 times.

V. Prospects for the Future of TypeScript

(I) The Development Trend of Language Features

Currently, the development speed of ECMAScript has slowed down. Community feedback shows that people are more concerned about scalability and performance rather than new fancy features of the type system. The TypeScript team will pay attention to the work of the ECMAScript committee, properly handle new features in the type system, and at the same time think about the impact of a 10-fold speed increase of the type checker and explore new possibilities.

(II) The Possibility of Combining with Artificial Intelligence

Use the fast type checker to provide context information for large language models (LLMs), such as type resolution results, symbol declaration locations, etc. Check the output of AI in real-time to ensure its semantic correctness and provide a guarantee for AI to generate safe and reliable code, opening up new development paths.

(III) The Conception of a Native Runtime

Explore whether a native runtime for TypeScript is possible. Currently, there is Deno written in Rust. Although there are some factors in JavaScript that affect performance, such as the object model and the way of handling numbers, creating a native runtime for TypeScript faces many uncertain factors, and the future development direction is still unclear.

VI. Third-Party Contributions and Community Impact

The transition from JavaScript to Go is relatively gentle for the system. Although there are fewer people who know both Go and JavaScript compared to those who only know JavaScript, which may lead to a decrease in the number of contributors, the number of people who contribute to the compiler was originally not large, and they are usually interested in stepping into the native environment. The Go language is simple, and its simple design has brought remarkable results such as a 10-fold performance improvement, which will not hinder the vitality and development of the community.

VII. Comparison of Common Statements between TypeScript and the Go Language

(I) Loops

TypeScript (Based on JavaScript)

for loop:

for (let i = 0; i < 10; i++) {
    console.log(i);
}

for...of loop (used to iterate over iterable objects, such as arrays):

const arr = [1, 2, 3];
for (const num of arr) {
    console.log(num);
}

for...in loop (mainly used to iterate over the properties of an object):

const obj = { a: 1, b: 2 };
for (const key in obj) {
    console.log(key, obj[key]);
}

Go Language

for loop (The Go language has only one basic loop structure, the for loop, but it can implement various loop ways):

for i := 0; i < 10; i++ {
    fmt.Println(i)
}

Iterate over iterable objects such as arrays and slices:

arr := []int{1, 2, 3}
for index, value := range arr {
    fmt.Println(index, value)
}

Iterate over a map:

m := map[string]int{"a": 1, "b": 2}
for key, value := range m {
    fmt.Println(key, value)
}

(II) Functions

TypeScript

Function definition:

function add(a: number, b: number): number {
    return a + b;
}

Arrow function:

const multiply = (a: number, b: number): number => a * b;

Go Language

Function definition:

func add(a int, b int) int {
    return a + b
}

Anonymous function (can be assigned to a variable or passed as a parameter):

multiply := func(a int, b int) int {
    return a * b
}

(III) Object-Oriented Programming (OOP)

TypeScript

Class definition:

class Animal {
    name: string;
    constructor(name: string) {
        this.name = name;
    }
    speak() {
        console.log(`${this.name} makes a sound.`);
    }
}

Inheritance:

class Dog extends Animal {
    constructor(name: string) {
        super(name);
    }
    speak() {
        console.log(`${this.name} barks.`);
    }
}

Go Language

The Go language does not have the traditional concepts of classes and inheritance. It achieves functions similar to OOP through structs and method sets.
Struct definition:

type Animal struct {
    Name string
}
func (a *Animal) Speak() {
    fmt.Printf("%s makes a sound.\n", a.Name)
}

Realize something similar to inheritance through composition:

type Dog struct {
    Animal
}
func (d *Dog) Speak() {
    fmt.Printf("%s barks.\n", d.Name)
}

(IV) Functional Programming

TypeScript

Example of a higher-order function (accepts a function as a parameter):

function operateOnArray(arr: number[], callback: (num: number) => number): number[] {
    return arr.map(callback);
}
const result = operateOnArray([1, 2, 3], num => num * 2);

Immutable data structures can be implemented with the help of external libraries (such as Immutable.js). For example:

import { fromJS } from 'immutable';
const list = fromJS([1, 2, 3]);
const newList = list.push(4);

Go Language

Example of a higher-order function:

func operateOnSlice(slice []int, callback func(int) int) []int {
    result := make([]int, len(slice))
    for i, v := range slice {
        result[i] = callback(v)
    }
    return result
}
result := operateOnSlice([]int{1, 2, 3}, func(num int) int {
    return num * 2
})

The Go language itself does not natively support immutable data structures like some functional programming languages. However, immutable behavior can be simulated through some design patterns and libraries, such as ensuring data immutability by copying structs and other methods.

Reference: https://www.youtube.com/watch?v=pNlq-EVld70&ab_channel=MicrosoftDeveloper