Modernizing Norma: HAL-Based Architecture

Hey guys! Let's talk about something super cool: how we can make Norma, our standard library, way better and easier to work with. We're going to dive into a new architecture, a HAL-based approach, which stands for Hardware Abstraction Layer. This is going to make Norma more efficient, maintainable, and just plain awesome. We'll be looking at the problems we're facing now, the solutions we're proposing, and how it's all going to come together. So, buckle up, because we're about to get technical, but in a fun and understandable way!

The Current State of Norma: Problems and Challenges

Right now, the Norma standard library uses template substitution with a massive registry. Think of it like a giant map that tells the system how to translate things. Here's the kicker: this map is huge and complex. We've got fons/norma/index.json with a whopping 32,000 tokens of template mappings. That's a lot of data! And then there's opus/rivus/.../norma-registry.gen.ts, which uses 49,000 tokens of if/else chains. These chains are basically long lists of conditions that determine how things are handled.

This current approach is, to put it mildly, a bit of a headache. First off, it's tough to maintain. Templates are strings, which means no handy IDE support to help you write code. You don't get the benefits of type checking to catch errors early. Second, there's a divergence between faber and rivus. Faber reads JSON, while rivus uses generated if/else chains. This means that changes need to be made in multiple places, increasing the chance of errors. Finally, it's complex to extend. Want to add a new function? You have to update the registry for all the targets. This is not only time-consuming but also opens up opportunities for mistakes.

The HAL-Based Solution: A New Architecture for Norma

To solve these problems, we're going to switch things up. We're replacing the old system with two main patterns: HAL modules and primitive types. This change makes things simpler and more effective. It also allows us to build a single source of truth.

1. HAL Modules: Building Blocks for Importable Modules

First up, we have HAL modules. These are for importable modules like console, process, filesystem, and network. These are essential parts of any system. Imagine this: We create a file like norma/hal/consolum.fab. In this file, we define how the console module works, using a special syntax that tells the system what to do. Take a look at this example code snippet:

# norma/hal/consolum.fab
@ subsidia ts "codegen/ts/consolum.ts"
@ subsidia go "codegen/go/consolum.go"
pactum consolum {
    @ externa
    functio fundeLineam(textus msg) -> vacuum
}

In this code:

• @ subsidia specifies that this is a HAL module and where to find the native implementations for different targets (like TypeScript and Go).
• @ externa marks methods as externally implemented. This tells the system that the actual code for this method is not in the .fab file, but in a separate file generated for a specific target.
• pactum defines the module consolum and the function fundeLineam (which, in this case, would print a line of text).

The key here is that we'll have native implementations in codegen/<target>/ subdirectories. This means real code that can be tested, type-checked, and have IDE support. The codegen system will resolve imports to these native implementation files. This keeps things organized and makes it easy to work with.

2. Primitive Types: Core Building Blocks

Next, we have primitive types. These are for built-in types such as lista (list), tabula (table), textus (text), numerus (number), and so on. These are the fundamental data types that everything else is built upon. Here's an example:

# norma/innatum/lista.fab
@ innatum ts "Array"
@ innatum py "list"
@ innatum rs "Vec"
pactum lista<T> {
    @ verte ts "push"
    @ verte py "append"
    functio adde(T elem) -> vacuum
    
    @ verte ts "length"
    @ verte py "len"
    functio magnitudo() -> numerus
}

In this example:

• @ innatum maps the pactum to native types for each target (TypeScript uses Array, Python uses list, and Rust uses Vec).
• @ verte maps methods to native method names (like push in TypeScript and append in Python for adding an element to the list).

This approach means no wrapper classes and no runtime overhead. Both compilers parse the same .fab files, which keeps everything consistent and efficient. This also improves compiler parity, as both faber and rivus now parse the same files.

Directory Structure: Keeping Everything Organized

Here's how the directory structure will look, keeping everything organized and easy to navigate:

fons/norma/
├── hal/
│   ├── consolum.fab
│   ├── processus.fab      # process
│   ├── tabularium.fab      # filesystem
│   ├── ...
│   └── codegen/
│       ├── ts/
│       │   ├── consolum.ts
│       │   └── processus.ts
│       └── go/
│           ├── consolum.go
│           └── processus.go
├── innatum/
│   ├── lista.fab
│   ├── tabula.fab
│   ├── textus.fab
│   ├── numerus.fab
│   ├── fractus.fab
│   └── copia.fab
└── index.fab               # re-exports (optional)

This structure makes it super easy to find what you're looking for. The hal directory contains all the HAL modules, while innatum has all the primitive type definitions. The codegen directory holds the native implementations for each target language. This structure supports self-documenting as the .fab files are also the API documentation.

Benefits: Why This New Architecture Rocks!

So, why are we doing all of this? Here are the key benefits:

1. Single Source of Truth: All the declarations are in .fab files, which means we have one place to look for information, not scattered across JSON files and generators.
2. Compiler Parity: Both faber and rivus will parse the same files, so the behavior will be consistent across different compilers.
3. Testable: Native implementations are real code, making them easy to test and verify.
4. Maintainable: Adding a new function is as simple as adding a method to the .fab file and implementing it in the codegen/ directory.
5. Self-Documenting: The .fab files themselves serve as the API documentation, so you can easily understand how everything works.

Migration: The Path Forward

Alright, let's talk about how we're going to get there. The migration plan is as follows:

1. Implement @ subsidia codegen support (see issue #165).
2. Implement @ innatum / @ verte for primitives.
3. Create the norma/innatum/ directory with primitive declarations.
4. Migrate HAL modules incrementally (consolum and processus are already done).
5. Deprecate and remove norma/index.json and the registry generator.

This incremental approach will allow us to roll out the changes gradually, ensuring that everything works smoothly.

Related Issues: Diving Deeper

If you want to learn more and get involved, check out these related issues:

• #165 - Implement @ subsidia codegen – This is the key to supporting HAL modules.
• #163 - Triple-quote multi-line strings (for help text in HAL) – This will allow us to include detailed help text in our HAL modules.

This is a major step forward for Norma. By adopting this new HAL-based architecture, we're making our standard library more robust, more maintainable, and much more enjoyable to work with. Thanks for reading, and let's keep making Norma the best it can be!