Models and data

A model from a spec

mctl gen model user.api writes a module whose tables are #orm annotations — one struct per type block:

$ mctl gen model user.api --dir out
mctl: wrote 4 file(s) from user.api

// Code generated by moonctl from the `.api` spec. DO NOT EDIT.

///|
/// `User` table, from the `.api` `type` block of the user service.
#orm.table(name="user")
pub(all) struct User {
  #orm.column(primary_key=true, autoincrement=true)
  id : Int64?
  name : String
  #orm.column(kind="int")
  age : Int32
  #orm.skip
  tags : Array[String]
}

The annotation says what the type cannot: id is the key, and an optional integer key is autoincrement; Int32 names its SQL kind, because Int64 is a bigint and Int32 is not; #orm.skip keeps a field with no single column — a slice, a map, a nested type — out of the table, filling it from Default when a row is read.

The model itself is not written here. moon.mod carries one rule, and moonorm's own generator reads the structs on every build:

rule(
  name: "orm",
  command: "moonx moonbitstack/moonorm/cmd/orm@0.11.7 -C \"$mod_dir\" \"$input\" -o \"$output\"",
)

dev_build(rule: "orm", input: "models.mbt", output: "models.g.mbt")

formatter(ignore: [ "models.g.mbt" ])

What comes out is User::model() — the columns, the row decoder, the INSERT projection. The migration pair is generated beside it and reads through that model, so it follows the annotation rather than a copy of it: user_up creates the table, user_down drops it.

--flat writes the older single file, user_model.mbt, with the model spelled out by hand — for a project with no module of its own.

A model from DDL

mctl gen crud schema.sql reads CREATE TABLE statements and writes the same tree, plus the calls for the table:

CREATE TABLE `orders` (
  `id` BIGINT NOT NULL,
  `user_id` BIGINT NOT NULL,
  `total` DOUBLE NOT NULL,
  `note` VARCHAR(200) NULL,
  `created_at` TIMESTAMP NOT NULL,
  PRIMARY KEY (`id`)
);

///|
/// `orders` table.
#orm.table(name="orders")
pub(all) struct Orders {
  #orm.column(primary_key=true)
  id : Int64
  user_id : Int64
  total : Double
  note : String?
  created_at : String
}

A nullable column is an optional field, which is how #orm spells NULL — and the CRUD helpers bind None as SQL NULL rather than as a value of the column's type. A DEFAULT travels onto the annotation as SQL.

And one function per call the table can answer, each reading through the model:

pub fn orders_insert(sess : @moonorm.Session, record : Orders) -> @moondb.ExecResult raise @moondb.DbError
pub fn orders_all(sess : @moonorm.Session) -> Array[Orders] raise @moondb.DbError
pub fn orders_find_by_id(sess : @moonorm.Session, id : Int64) -> Orders? raise @moondb.DbError
pub fn orders_update(sess : @moonorm.Session, record : Orders) -> @moondb.ExecResult raise @moondb.DbError
pub fn orders_delete_by_id(sess : @moonorm.Session, id : Int64) -> @moondb.ExecResult raise @moondb.DbError

The lookup is named after the primary key the DDL declared — find_by_id here, find_by_code for a table keyed on code.

A live database

A schema is often in the server, not in a file. model mysql datasource and model pg datasource connect through moondb, read the column list and types, and write the model from what is actually there:

$ mctl model mysql datasource root:pass@tcp(127.0.0.1:3306)/shop
mctl: generated mysql_model.mbt

$ mctl model pg datasource postgres://app:pass@localhost:5432/shop
mctl: generated pg_model.mbt

Both need a reachable server: the command opens a connection, asks the catalogue for the tables and columns, and closes it. Nothing else in moonctl needs a database.

Documents

mctl model mongo user.api treats each type block as a document. The generated file carries the document and a typed repository, over one port that names the five calls a repository makes:

pub(all) struct Collection {
  name : String
  run : (MongoOp, Json, Json) -> Json raise MongoError
}

#derive(FromJson, ToJson)
pub(all) struct LoginReqDoc {
  username : String
  password : String
}

pub fn LoginReqRepo::find(
  self : LoginReqRepo,
  username : String
) -> LoginReqDoc raise MongoError {
  let reply = (self.coll.run)(
    FindOne,
    filter_of("username", @json.to_json(username)),
    empty_doc(),
  )
  match reply {
    Null => raise MongoMissing(collection=self.coll.name, key=username.to_string())
    _ =>
      @json.from_json(reply) catch {
        e => raise MongoBadDocument(collection=self.coll.name, why=e.to_string())
      }
  }
}

A driver binds run to insert-one / find-one / find / replace-one / delete-one and the repositories work. insert and all exist for every document; find, replace and delete appear when the type has a field to look a record up by. A field that names another type is stored as that type's document, so owner: User becomes owner : UserDoc. The file needs moonbitlang/core/json and no driver.

A cache in front

mctl model cache user.api writes the cache-aside layer for the same tables: read through on find, invalidate on write and on evict.

pub(all) struct Cache[T] {
  get : (String) -> T? raise CacheError
  set : (String, T) -> Unit raise CacheError
  del : (String) -> Unit raise CacheError
}

pub fn login_req_find_cached(
  cache : Cache[LoginReq],
  username : String,
  load : (String) -> LoginReq? raise CacheError,
) -> LoginReq? raise CacheError {
  let key = login_req_cache_key(username)
  let hit : LoginReq? = (cache.get)(key) catch { _ => None }
  match hit {
    Some(record) => Some(record)
    None =>
      match load(username) {
        Some(record) => {
          (cache.set)(key, record) catch { _ => () }
          Some(record)
        }
        None => None
      }
  }
}

The port is generic, so the cache holds the record rather than the text it serialises to, and an in-memory implementation is generated alongside — memory_cache[T](), a Map behind the three calls. A cache that fails is a miss and a failed invalidation is swallowed: a broken cache costs latency, never a request.

Migrations

mctl migrate user.api writes the SQL a spec implies, one table per type block, with a primary key where a field is named id:

CREATE TABLE IF NOT EXISTS `profiles` (
  `id` BIGINT NOT NULL,
  `name` VARCHAR(191) NOT NULL,
  PRIMARY KEY (`id`)
);

CREATE TABLE IF NOT EXISTS `login_reqs` (
  `username` VARCHAR(191) NOT NULL,
  `password` VARCHAR(191) NOT NULL
);

A spec says what a value looks like, not what the database should enforce, so the file carries columns and nothing else — indexes, uniqueness and foreign keys are yours to add. The mapping:

Where next

A .proto can become a service, and an HTTP gateway in front of it. That is the gateway page.