particularly in rust, basically every data structure in the compiler's middleend ends up looking like a relational database table—i.e. you replace pointers with typed indices into the table—and a lot of the analysis work boils down to doing complicated recursive queries on those tables
most of rustc's IRs use some kind of index-as-pointer representation, the austral compiler uses CRUD-style tables in the middleend, and @welltypedwit.ch is apparently doing something similar with polaris
don't quote me on this but i vaguely remember a talk by chandler carruth where he talks about how the carbon compiler uses this kind of index-based design to avoid unnecessary pointer overhead and improve cache locality
but yeah the idea in vega is basically to treat everything as a ~whole program compiler that works on a graph of declarations with the catch being that instead of throwing everything away each time and starting from scratch, it saves all intermediate results in a DB (e.g. types of declarations)
i haven't had much time to work on it so it's still pretty early, but basically: files don't really mean much and instead everything is represented as a graph of declarations that is stored in a database.
so when something in the source file changes, that declaration (not file) and all its reverse dependencies are marked as dirty and recompiled (possibly in parallel, even with other passes on the same declaration).
so any intermediate result (e.g. "the type of map is ...") is persisted in the DB and only recomputed when necessary.
ideally, you would get the benefits of separate compilation (and more) while having as much access to cross-module information as a whole program compiler
Comments
polaris' module system is a hacky, naive mess that i really need to fix sometime
ideally, you would get the benefits of separate compilation (and more) while having as much access to cross-module information as a whole program compiler
I don't agree with the authors 1st conclusion, but do agree with the 2nd. ∴file≡program
https://euromathsoc.org/magazine/articles/205