The Cloesce Schema Language

Cloesce is a schema language that describes a full stack application built on Cloudflare’s edge ecosystem.

Note

Cloesce is under active development, expanding its feature set as it pushes toward full Cloudflare support across any language.

The syntax and features described here are subject to change as the project evolves.

Contributing

Contributions are welcome at all levels. Join our Discord to discuss ideas, report issues, or get help getting started. Create an issue on GitHub if you find a bug or have a feature request.

Coalesce

Check out Coalesce, an accelerated web app framework for Vue.js and Entity Framework by IntelliTect. Many core concepts of Cloesce come directly from Coalesce (Cloesce = Cloudflare + Coalesce).

Resources

LLMs

Interact with this documentation with an LLM by utilizing the llms-full.txt found here. Download from the terminal using curl:

curl https://cloesce.pages.dev/llms-full.txt -o llms-full.txt

Examples

There are several example projects available in the GitHub repository, which can be found here.

VS Code Extension

A basic language highlighting extension for Cloesce is available in the VS Code marketplace.

In the future, this extension will also include an LSP server.

Getting Started

Tip

Cloesce runs on Cloudflare Workers. Familiarity with Workers and Wrangler is recommended. If you are new to Workers, check out the official Cloudflare Workers documentation.

Welcome to the Getting Started guide. This guide covers:

• Installing Cloesce
• A basic project structure with create-cloesce
• Building, migrating, and running your application

Installation

Note

Only TypeScript-to-TypeScript compilation is currently supported.

Installing the Compiler

Linux and macOS

curl -fsSL https://cloesce.pages.dev/install.sh | sh

Windows (PowerShell)

irm https://cloesce.pages.dev/install.ps1 | iex

Then verify the installation:

cloesce version

Starting a New Project

The fastest way to get a Cloesce project up and running is to use the create-cloesce template.

Prerequisites

1. Sign up for a Cloudflare account (not necessary for local development)
2. Install Node.js (version 16.17.0 or later)

create-cloesce

Run the following command in your terminal:

npx create-cloesce my-cloesce-app

After running the command, navigate into your new project directory:

cd my-cloesce-app

A simple project structure is created for you.

├── src/
│   ├── api/            # API route handlers
│   ├── web/            # Frontend web assets
│   └── schema/
│       └── schema.clo  # Cloesce schema
├── test/               # Example unit tests
├── migrations/         # Database migration files
├── cloesce.jsonc       # Cloesce configuration

Building and Migrating

Configuration

Define a cloesce.jsonc file in your project root to configure the Cloesce compiler:

{
  "src_paths": ["./src/schema"],
  "workers_url": "http://localhost:5000/api",
  "wrangler_config_format": "jsonc" // or "toml"
}

All keys are optional except src_paths, which tells the compiler where to find your .clo files:

Tip

Multiple configuration files can be defined for different environments:

• <name>.cloesce.jsonc

Select the desired configuration file using --env <name> when running Cloesce commands:

# given `staging.cloesce.jsonc` exists
cloesce --env staging ...

Compilation

In your root directory, run the following command to compile your schema:

cloesce compile

Important

Any generated artifacts should not be modified directly or committed to source control.

Import them into your backend and client code, relying on a cloesce compile build step to keep up to date with your schema.

Migrations

Tip

Schema modifications to a SQLite backed Model should be accompanied by a new migration. This ensures that your database schema stays in sync with your Cloesce Models.

Migrations turn a Cloesce schema into a set of SQL statements that can be applied to a database, tracking changes over time.

Each migration is written to <migrations_path>/<binding>/, and one of --binding or --all is required.

Specific Binding

cloesce migrate --binding <binding> <migration-name>

All Bindings

cloesce migrate --all <migration-name>

Apply D1 Migrations

Cloesce generates the SQL for migrations, but does not apply them.

If a D1 database is being utilized, a .sql file is generated, which you must apply using the Wrangler CLI:

npx wrangler d1 migrations apply <binding-name>

Apply Durable Object Migrations

A Durable Object’s SQLite storage is not reachable from the Wrangler CLI, so its migrations are generated as .ts modules instead of .sql files.

Import each one and pass it to durable. They are applied once, in order, before the Durable Object serves any request:

import { createApp, CfEnv } from "@cloesce/backend.js";
import initMigration from "../migrations/MyDo/1785712992_init.js";

export class MyDo extends DurableObject<CfEnv> {
  private base = createApp().durable(this, [initMigration]);

  async fetch(request: Request): Promise<Response> {
    return this.base.run(request);
  }
}

Running

After compilation and migrations, run your application locally with Wrangler:

npx wrangler dev --port <port-number>

Deploying

Deploy your application to Cloudflare’s edge with Wrangler:

npx wrangler deploy

Type Reference

This section provides a reference for the types available in the Cloesce Schema Language.

All Types

Primitives

Generics

Objects

Any Model or Plain Old Object defined in your schema can be used as a type:

model User for Db {
    primary {
        id: int
    }

    column {
        name: string
    }
}

poo Profile {
    user: User // A reference to the User Model
    bio: string
}

SQLite Compatible Types

Some areas of the schema will only accept types that are compatible with SQLite. By default, all of these types are NOT NULL in a SQLite database.

To allow NULL values, wrap the type in the option generic, e.g. option<string> (which is SQLite compatible).

Environment Declarations

Environment bindings are how Cloesce manages, references, and injects Cloudflare Workers bindings in your application.

Currently, Cloesce supports D1, KV, R2, Durable Objects, and Wrangler Environment Variables.

By defining these bindings in your schema, you enable Cloesce to:

• Describe Models composed of data stored in these bindings.
• Generate a Wrangler configuration for each binding.
• Generate a fully typed interface for interacting with each binding in your application code.
• Handle database migrations

Tip

In this alpha, any top level declaration in Cloesce is global across any file in the project.

This means that environment bindings declared in one file can be referenced and used in any other file.

Workers KV and R2

Note

Cache control directives and expiration times are planned for a future release.

Workers KV

Cloudflare KV is a globally distributed key-value store.

Define a KV binding in your schema to generate a matching Wrangler configuration and a fully typed interface for querying that namespace in your application code.

kv MyNamespace {
    settings -> json {}

    // accept any number of parameters
    session -> SessionToken {
        token: string
    }

    // write custom key templates
    custom -> string {
        param1: string
        param2: string

        "path/to/{param1}/{param2}"
    }
}

A Wrangler configuration will be generated:

[[kv_namespaces]]
binding = "MyNamespace"
id = "replace_with_MyNamespace_id"

R2

Define a Cloudflare R2 binding in your schema to generate a matching Wrangler configuration and a fully typed interface for querying that bucket in your application code.

r2 MyBucket {
    getObject {}

    getObjectCustomKey {
        param1: string
        param2: string

        "path/to/{param1}/{param2}"
    }
}

Unlike Workers KV, R2 bindings do not have a return type, because they will always return the Cloudflare R2Object (a HEAD request to the object, not the full value).

Additionally, a Wrangler configuration will be generated:

[[r2_buckets]]
binding = "MyBucket"
bucket_name = "replace-with-mybucket-name"

D1

Cloudflare D1 is a distributed SQL database built on SQLite for Workers.

Define any number of D1 databases in your schema, and create Models to represent tables in those databases.

Defining a D1 Binding

To define a D1 environment binding, use the d1 block. Any number of bindings may be listed inside it:

d1 {
    MyDb
}

Wrangler Configuration

A Wrangler configuration will be generated for each D1 binding defined in the schema:

[[d1_databases]]
binding = "MyDb"
database_id = "replace_with_MyDb_id"
database_name = "replace_with_MyDb_name"
migrations_dir = "./migrations/MyDb"

Migrations

To generate SQL migration files for a specific binding, run the following command:

cloesce migrate --binding <binding> <migration-name>

Apply the generated migrations for a D1 database using the Wrangler CLI:

npx wrangler d1 migrations apply <binding-name>

Durable Objects

Cloudflare Durable Objects provide a way to run stateful code on Cloudflare’s edge network.

To describe them simply (a task difficult to do justice), Durable Objects are:

1. A place to store data (SQLite and KV storage).
2. A single threaded sequential execution context.
3. Capable of being sharded across any number of instances (think database-per-X).

Cloesce provides first class support for Durable Objects:

• Define DOs in your schema
• Generate a fully typed interface
• Use them as a Model backing
• Execute API methods within a DO’s context

Warning

Cloesce is only capable of using the modern SQLite backed Durable Objects, and does not support the legacy Durable Object storage API.

Defining a Durable Object Binding

To define a Durable Object environment binding, use the durable block:

durable MyShardedDo {
    shard {
        tenant: int
    }

    settings -> json { }

    userMap -> json {
        userId: int
        "user/{userId}"
    }
}

durable MyGlobalDo {
    settings -> json { }
}

The above example defines two Durable Object bindings:

• MyShardedDo: Any number of Durable Object instances can be created with different shard parameters. In this case, the tenant parameter is used to shard the Durable Object by tenant ID.

• MyGlobalDo: A singleton Durable Object that will always route to the same instance.

In both bindings, KV templates can be defined to generate a typed interface for interacting with the Durable Object’s KV storage.

Extending the Durable Object Class

The Cloesce Router will forward HTTP requests bound for a particular Durable Object from the Worker to the fetch method of the generated Durable Object class.

To implement custom logic for handling these requests, extend the generated Durable Object class and implement the fetch method:

import { createApp, CfEnv } from "@cloesce/backend.js";
import initMigration from "../migrations/SubRedditDo/1785712992_init.js";

export class SubRedditDo extends DurableObject<CfEnv> {
  private base = createApp().durable(this, [initMigration]);

  async fetch(request: Request): Promise<Response> {
    return this.base.run(request);
  }
}

See Building and Migrating for how those migration modules are generated.

Wrangler Configuration

A Wrangler configuration will be generated for each Durable Object binding defined in the schema:

[[durable_objects.bindings]]
class_name = "MyShardedDo"
name = "MyShardedDo"

[[durable_objects.bindings]]
class_name = "MyGlobalDo"
name = "MyGlobalDo"

[[migrations]]
new_sqlite_classes = [
    "MyShardedDo",
    "MyGlobalDo",
]
tag = "v1"

Environment Variables

Any number of environment variables can be defined in the schema, which will be placed in the Wrangler configuration and made available to the Worker at runtime.

Defining Environment Variables

Note

Variables are restricted to the same set of primitive types as SQLite Types.

To define an environment variable, use the var block in the schema:

var {
    MY_VAR: string
    MY_OTHER_VAR: int
}

Inject them into an API endpoint like so:

api Foo  {
    get bar -> string {
        inject { MY_VAR }
    }
}

Models

A Model in Cloesce defines a structure hydrated from stores of persistent data, such as:

• D1 databases
• Durable Objects
• KV namespaces
• R2 buckets.

Models do not exist in just one layer of your full stack application: they are a first class citizen across the frontend, backend, and database layers of your application.

This chapter will cover how to define Models that utilize Environment Bindings, and the relationships that can be defined between Models.

Worker Backed Models

A Model backed by no environment binding is referred to as a “Worker Backed Model”, or sometimes “a backing-less Model”.

Their fields are sourced from route parameters in a request URL.

For example:

model Gnat {
    route {
        id: int
        buzzing: bool
    }
}

The Gnat Model sources all data from an incoming request URL, and then disappears after the request is complete. It has no backing store, and no persistence.

Tip

Worker Backed Models can have any number of KV or R2 fields, using the route parameters to hydrate those fields. You can even have navigation fields!

SQLite Backed Models

A Model can be backed by a SQLite database, stored in either a D1 database or a Durable Object.

Defining an Environment Binding

To back a Model with a SQLite database, you first need some storage binding that supports SQLite.

// Cloudflare D1
d1 {
    MyDb
}

// Durable Object
durable MyDurableObject {
    shard {
        tenant: string
    }
}

Defining a Model

With D1

d1 {
    MyDb
}

model User for MyDb {
    primary {
        id: int
    }

    column {
        name: string
    }
}

The above code defines a Model “User” stored in the D1 database MyDb, with several properties:

With Durable Objects

durable MyDurableObject {
    shard {
        tenant: string
    }
}

model User for MyDurableObject::tenant {
    primary {
        id: int
    }

    column {
        name: string
    }
}

The above code defines a Model “User” stored in the Durable Object MyDurableObject, with several properties:

Tip

You may alias shard keys to any name in the Model declaration:

model User for MyDurableObject::tenant(alias) {
  // ...
}

If there are multiple shard keys, any number of them can be aliased:

model User for MyDurableObject::{tenant, org(alias)} {
  // ...
}

Tip

Just because a Model is backed by a Durable Object does not mean it uses the Durable Object’s SQLite storage.

A column, primary or foreign field must be defined for the Model to be represented as a table in SQLite.

For example, the Gnat Model from the previous chapter could be backed by a Durable Object:

model Gnat for MyDurableObject::tenant {
    route {
        id: int
        buzzing: bool
    }
}

Gnat’s fields are still ephemeral, existing for the duration of a request. However, it is tied to an instance of a Durable Object, which will be created based on the tenant shard key.

Across the Stack

Once defined, the User Model is a first class citizen across the frontend, backend, and database layers of your application.

For example, the backend of your application will generate the following TypeScript type for the User Model:

// .cloesce/backend.ts
export interface User {
  id: number;
  name: string;

  // iff backed by a Durable Object
  tenant: string;
}

In SQLite, the User Model will be represented as a table:

CREATE TABLE User (
    id INTEGER PRIMARY KEY,
    name TEXT
);

SQLite Column Constraints

Tip

All fields of a SQLite backed Model must be a SQLite compatible type.

This chapter provides a reference for the SQLite specific features of Models.

Primary Key

A primary block is required in every SQLite backed Model. It directly translates to the SQLite PRIMARY KEY constraint.

model User for Db {
    primary {
        id: int
    }
}

By default, primary keys are NOT NULL, UNIQUE, and AUTOINCREMENT (for integer fields).

Composite Primary Key

Any number of fields can be in a single primary block, or spread across any number of primary blocks.

For example, the following User Model has a composite primary key consisting of an id field and an email field:

model User for Db {
    primary {
        id: int
        email: string
    }
}

// Equivalent to:
model User for Db {
    primary {
        id: int
    }

    primary {
        email: string
    }
}

Foreign Key

The foreign block allows you to define foreign key relationships between Models, if they are in the same SQLite backing store.

It translates to the SQLite FOREIGN KEY constraint.

model Dog for Db {
    primary {
        id: int
    }
}

model Person for Db {
    primary {
        id: int
    }

    // Person has a foreign key relationship to Dog's field `id`
    // through its own field `dogId`.
    foreign Dog::id {
        // Types are inferred from the referenced field, so `dogId` is of type `int`.
        dogId
    }
}

Foreign key fields inherit the type of the field they reference. In the above example, Person::dogId is of type int because it references Dog::id, which is of type int.

Foreign key fields are NOT NULL by default.

Optional Foreign Key

To allow NULL values in a foreign key field, use the option modifier:

model Person for Db {
    primary {
        id: int
    }

    foreign Dog::id option {
        dogId
    }
}

Composite Foreign Key

A Model can have a composite primary key by listing multiple fields in a primary block.

Similarly, a Model can have a composite foreign key by listing multiple fields in a foreign block.

model Person for Db {
    primary {
        firstName: string
        lastName: string
    }
}

model Dog for Db {
    primary {
        id: int
    }

    foreign Person::{ firstName, lastName } {
        ownerFirstName
        ownerLastName
    }
}

Foreign Primary Key

A field can be both a primary key and a foreign key at the same time. This is useful for representing many-to-many relationships:

model Enrollment for Db {
    primary {
        foreign Student::id {
            studentId
        }

        foreign Course::id {
            courseId
        }
    }
}

model Student for Db {
    primary {
        id: int
    }
}

model Course for Db {
    primary {
        id: int
    }
}

Unique Constraint

The unique tag adds a unique constraint over one or more existing fields on a Model. It translates to the SQLite UNIQUE constraint. A field may participate in any number of unique constraints.

[unique email, profileId, dogId]
[unique username]
[unique dogId]
model User for Db {
    primary {
        id: int
    }

    column {
        email: string
        username: string
    }

    foreign Profile::id {
        profileId
    }

    foreign Dog::id {
        dogId
    }
}

KV Fields

Any Model may have any number of Cloudflare KV hydrated fields.

KV fields reference templates defined in a KV bindings or Durable Object bindings.

Defining a KV Field

A field in a Model can be hydrated from KV by referencing a binding defined on a kv namespace:

kv MyNamespace {
    settings -> json { }
}

model User {
    kv MyNamespace::settings {
        settings
    }
}

The above snippet defines a Model User with a KV field settings that is sourced from the namespace MyNamespace under the static key "settings".

The value in the template is typed as json, and Cloesce will automatically handle the serialization and deserialization of this field when reading from and writing to KV.

Note

To use a Durable Object’s KV field, shard fields must be provided in the kv field:

durable MyDurableObject {
    shard {
        tenant: string
    }

    settings -> json { }
}

model User for MyDurableObject::tenant {
    kv MyDurableObject::{settings, tenant} {
        settings
    }
}

Key Interpolation

A common pattern is to format a key such that any number of related values can be stored under that template. For example:

kv MyNamespace {
    profile -> json {
        userId: int

        "profile/{userId}"
    }

    profileImplicitKey -> json {
        userId: int
    }
}

Here, profile accepts one parameter, userId. The key for this field in KV is defined as "profile/{userId}", where {userId} is a placeholder replaced with the actual value of the userId parameter when accessing KV.

In the profileImplicitKey field, the key is not explicitly defined, so Cloesce will automatically generate a key based on the field name and its parameters. In this case, the key will be "profileImplicitKey/userId/{userId}".

Any column or route field on a Model can be used to populate the parameters of a KV field, as long as the types match. For example:

model User for MyDb {
    primary {
        id: int
    }

    column {
        friendId: int
    }

    kv MyNamespace::profile(id) {
        profile
    }

    kv MyNamespace::profile(friendId) {
        friendProfile
    }
}

R2 Fields

You can easily integrate Cloudflare R2 into your application by defining R2 fields in your Models.

Read the R2 Bindings section in the Environment chapter for more information on how to define an R2 binding in your schema.

Defining an R2 Field

Note

R2 is used to store large unstructured data.

Cloesce will not query or buffer the full value of an R2 field into the Worker runtime. Only a HEAD request is made to R2 to check for existence and retrieve metadata.

A field in a Model may reference an R2 bindings template to define an R2 field:

r2 MyBucket {
    image {
        key: string
        "images/{key}"
    }
}

model Image {
    route {
        id: string
    }

    r2 MyBucket::image(id) {
        my_image
    }
}

The above snippet defines a Model Image with an R2 field my_image that is stored in the bucket MyBucket under the key “images/{id}”, where {id} is a placeholder that will be replaced with the actual value of the id route field when accessing R2.

Navigation Fields

Data can exist in many different places. Modern ORMs easily answer the question:

• “Can I represent relationships between tables in the same database?”

But what about more complicated system designs?

• “Can a table have a relationship with a table in another database?”
• “…With a Durable Object?”
• “…KV and R2?? “
• “Do I even need SQL to have a relationship with data??!”

Through Navigation Fields and the Cloesce ORM, Cloesce can represent relationships between any Model, regardless of where the data is stored.

Defining Navigation Fields

A navigation field is a one or many relationship to another Model, which can be any Model (including itself).

Although the Cloesce ORM is able to operate with minimal information, the following MUST be provided to any navigation field declaration:

• Durable Object Shards
• Route fields

One-To-One

A one-to-one navigation is defined with the one keyword, and will only ever result in a single instance of the related Model being returned (or undefined if no related instance exists).

Classic Example

model Person for Db {
    primary {
        id: int
    }

    foreign Dog::id {
        dogId
    }

    one Dog::id(dogId) {
        dog
    }
}

model Dog for Db {
    primary {
        id: int
    }
}

In the above example, the Person Model has one Dog Model, which will be populated by matching the Person’s dogId field with the Dog’s id field.

No Discriminator Example

It is not necessary to provide any discriminators to the navigation field, but it may produce a more efficient query plan to do so.

The following is also valid:

model Person for Db {
    primary {
        id: int
    }

    one Dog {
        dog
    }
}

// ...

Here, the Person Model has one Dog Model, but no discriminator is provided. This results in a far less efficient query plan, as the ORM will scan for the first Dog instance in the database, as opposed to the previous example which will search, utilizing the Dog’s primary key index to find the related instance.

One-To-Many

A many relationship suggests that any Model that matches the provided discriminator (or all instances if no discriminator is provided) will be returned in an array.

Classic Example

model Person for Db {
    primary {
        id: int
    }

    many Dog::ownerId(id) {
        dogs
    }
}

model Dog for Db {
    primary {
        id: int
    }

    foreign Person::id {
        ownerId
    }
}

The above example defines a relationship where Person has many Dogs, which will be populated by matching the Person’s id field with the Dog’s ownerId field (a search operation).

No Discriminator Example

Like with the one block, a discriminator is not required, but providing one may produce a more efficient query plan.

The following is also valid:

model Person for Db {
    primary {
        id: int
    }

    many Dog {
        dogs
    }
}
// ...

By default, fetching a Person will result in the dogs field containing every single dog in the database, because no discriminator was provided.

To Any Model?

Yes! Cloesce can represent and even hydrate navigation fields between any Model.

D1 to Durable Object Example

model PersonIndex for D1Db {
    primary {
        personId: int
        tenant: string
    }

    one Person::{personId, tenant} {
        person
    }
}

model Person for PersonDo::{personId, tenant} {
    kv PersonDo::{profile, personId, tenant} {
        profile
    }

    // We can even point back to the index Model!
    one PersonIndex::{personId, tenant} {
        index
    }

    // What if we also wanted all people in a tenant? We can do that too!
    many PersonIndex::tenant {
        allPeopleInTenant
    }
}

Here, the PersonIndex Model is backed by a D1 database, meaning it is a table PersonIndex in D1Db.

• Within every PersonIndex row is a logical navigation to the Person Model, which is backed by the Durable Object PersonDo.
• We must provide the shard values of personId and tenant to locate the correct PersonDo.

Worker Backed Example

It is not necessary for a Model to be backed by anything in order to have navigation fields.

model Logical {
    route {
        tenant: string
    }

    many DoBacked::tenant {
        allDoBackedInTenant
    }

    one Empty {
        empty
    }

    many Empty {
        empties
    }
}

model Empty {}

model DoBacked for Do::tenant {
    primary {
        id: int
    }
}

In this example, the Logical Model is not backed by any kind of database. It exists purely from values passed from HTTP requests (route fields).

For more details on hydration, see the ORM Chapter

Data Sources

Models can be composed of a lot of different kinds of data.

• You may want to retrieve only a subset of the relationships for a Model.
• You may want to write queries to filter, sort, order, paginate, and even authenticate and authorize access to data.

Unlike other ORMs, Cloesce is not a general purpose query builder.

Instead, when you need business logic, you define a Data Source: stubs implemented in the runtime that describe how to get, list, or save data for a Model, from any set of parameters.

This chapter provides a reference for how to write Data Sources in Cloesce, which are the building blocks for all data retrieval in your application.

Data Sources Overview

What are Data Sources?

Data Sources are Cloesce’s answer to querying when there is potential for:

• overfetching
• recursive relationships
• complex business logic

Every Data Source is composed of:

• an Include Tree
• get, list, and save operations

Data Sources are used extensively in the backend, but are also exposed to the client during API generation. A client may call any of the CRUD operations on a Data Source, making them the go-to method of writing any get, list or save operation for a Model.

Include Trees

To determine which fields to hydrate, Cloesce uses a construct called the Include Tree. An Include Tree is a recursive structure that represents the relationships between Models and their fields.

Consider the following example of a Person and Dog Model:

model Person for Db {
    primary {
        id: int
    }

    foreign Dog::id option {
        dogId
    }

    one Dog::id(dogId) {
        dog
    }
}

model Dog for Db {
    primary {
        id: int
    }

    foreign Person::id {
        ownerId
    }

    one Person::id(ownerId) {
        owner
    }
}

Person has one Dog, and Dog has one Person.

If we were to fetch naively, we would end up in an infinite loop of fetching Person and Dog instances. To prevent this, Cloesce will generate the following Default Data Source for the Person and Dog Models:

source Default for Person {
    include {
        dog
    }
}

source Default for Dog {
    include {
        owner
    }
}

Each branch of the Include Tree is a relationship that will be joined when fetching a Model. Relationships can be traversed from Model to Model: if I include an owner, I can now include any number of relationships that belong to the owner.

Default Include Tree

To prevent overfetching (and infinite loops), the Default Data Source will join:

• R2 and KV fields
• All One-to-One Navigation Fields
• The near side of all 1:M Navigation Fields
• The near side of a recursive relationship

CRUD Operations

Alongside the Include Tree, every Data Source has three operations: get, list, and save.

The default implementations of these operations are as follows:

• get: fetch a single instance by primary keys, route keys and shard keys
• list: fetch a list of instances by primary keys, route keys and shard keys via limited seek pagination
• save: insert, update or upsert an instance and all children by a partial snapshot of a Model

Each default implementation will follow the Include Tree defined in a Data Source, omitting any relationships not within the tree.

For example:

source Default for Person {
    include {} // Empty!
}

The above Data Source includes no relationships, so the default get and list operations will only return the Person’s primary keys and foreign keys, and will not join any relationships. save will simply no-op on children.

For more information on how the Cloesce ORM and Query Planner work, read the ORM Chapter.

Custom Data Sources

Default capabilities for every Data Source are provided by Cloesce, but they can be naive:

• Children are not limited to a certain number of results.
• many relationships are ordered descending by primary key.
• No filtering beyond what has been defined in the schema is provided.
• CRUD operations are exposed to the client.

Define custom Data Sources for any Model in your schema to implement the exact behavior you want for your application.

Defining a Data Source

Note

Any scalar property (i.e. SQLite columns) will be included in the Include Tree.

Data Sources can be defined with a source block.

In the inner include block, you can specify all relationships to include in that Data Source, including R2, KV, and Navigation Fields.

source WithDogsOwnersDogs for Person {
    include {
        dogs {
            owner {
                dogs {
                    // ... could keep going!
                }
            }
        }
    }
}

Overriding the Default Data Source

The Default Data Source for a Model can be overridden by giving a source block the name Default, changing the default behavior of that Model when it is hydrated without a specified Data Source:

// Override the default to be empty
source Default for Person {
    include {}

    // ...methods
}

Get Method

Each time Cloesce needs to hydrate an instance of a D1 backed Model, it requires a Data Source with a get method defined.

If you do not define a get method, Cloesce will use a default get-by-id implementation. Otherwise, you can define a custom get method on any Data Source:

source ByName for Person {
    get {
        name: string
    }
}

A backend stub will be generated for the get method above, which you can then fill with custom logic for fetching a Person by their name instead of their id.

The save and list methods will use default implementations if not overridden.

instance tag

Data Sources are used by API methods to denote how to hydrate an “instance method”. From the client’s perspective, an instance method is a method on a class, like:

const person = await Person.get({ id: 1 });
await person.instanceMethod();

How does Cloesce know which instance to call the method on? A naive Data Source could be defined like:

source ByName for Person {
    get {
        name: string
    }
}

which will result in the client expecting name to be passed on every instance method hydrated with ByName:

const person = await Person.get({ id: 1 });
await person.instanceMethod(person.name);

Because person already has the name field, it is redundant to require it to be passed in again. To tell Cloesce that a parameter is already available on the client instance (i.e. it is a field of the Model), you can use the instance tag in your get method parameters:

source ByName for Person {
    include {}

    get {
        [instance]
        name: string
    }
}

List Method

The get method of a Data Source is special in that it can be used to hydrate an instance of a Model on an API call (see instance methods).

The list method however is purely utility for the backend and client to retrieve a list of instances of a Model.

source ByName for Person {
    include {
        dogs
    }

    list {
        lastSeenName: string
        limit: int
    }
}

Internal Data Source

If a Model is decorated with the [crud] tag (see the CRUD Generation chapter), Cloesce will generate client methods for all Data Sources on that Model.

Data Sources are the preferred way to retrieve Models in Cloesce for both the backend and the client. However, you may not want to expose a Data Source to the client, and only use it internally in your backend.

Tag any Data Source with internal to prevent Cloesce from generating client methods for that Data Source:

[internal]
source InternalOnly for Person {
    // ...
}

APIs

Cloesce generates an RPC-like REST API for every Model in your application.

This chapter covers how to:

• Define REST APIs
• Generated CRUD APIs
• Inject dependencies into APIs
• Validate API inputs with Runtime Validation
• Define Plain Old Objects

REST APIs

By defining an API for a Model, you can specify REST endpoints that are generated as backend stubs and client methods, routed by the Cloesce runtime.

Defining an API

Given some Model, we can define an API for it like so:

model Person for Db {
    primary {
        id: int
    }
}

api Person {
    get byId -> Person {
      id: int
    }

    post create -> Person {
      name: string
    }

    delete del {
      id: int
      X_Auth_Token: string
    }

    put update {
      id: int
      name: string
    }

    patch updatePartial {
      id: int
      name: string
    }
}

The above code defines an API for the Person Model:

All of the above methods are static. They do not hydrate an instance of that Model implicitly.

Tip

It is heavily recommended to use Data Sources to define generic get, list and save methods instead of defining them in an API.

Every Data Source will generate a corresponding API method for the client by default.

[header] tag

API methods accept all parameters in the request body by default. If you want to accept a parameter from the request headers instead, you can tag that parameter with [header]:

api Person {
    delete del {
      id: int

      [header]
      X_Auth_Token: string
    }
}

Headers may use Pascal_Snake_Case to indicate that it should be parsed as X-Auth-Token in the request headers.

Generated Code

After running cloesce compile, the above API definition could be implemented in TypeScript as follows:

import { Api } from "@cloesce/backend.js";

export default {
  byId(id) {
    // ...
  },

  create(name) {
    // ...
  },

  del(id) {
    // ...
  },

  update(id, name) {
    // ...
  },

  updatePartial(id, name) {
    // ...
  },
} satisfies Api.Person.Of;

// alternatively, each API could be implemented individually:
export const byId: Api.Person.byId = (id) => {
  // ...
};

Like with most RPC frameworks, the API implementation must be registered so it can be dispatched to on a matching request. A missing implementation results in a 501 Not Implemented response.

import { CfEnv, createApp, Person } from "@cloesce/backend.js";
import person from "./person.js";

// src/index.ts
export default {
  async fetch(request: Request, env: CfEnv): Promise<Response> {
    const app = createApp().worker(env).register(Person, person);

    return app.run(request);
  },
};

Registering Durable Object APIs

Durable Objects receive forwarded requests from Workers by the Cloesce runtime, and need their own app registration:

import { createApp, CfEnv } from "@cloesce/backend.js";
import person from "./person.js";

export class MyDurable extends DurableObject<CfEnv> {
  private base = createApp().durable(this).register(Person, person);

  async fetch(request: Request): Promise<Response> {
    return this.base.run(request);
  }
}

Instance Methods

Many API endpoints start out by:

• Selecting a row from a database
• Seeing if it exists
• Returning a 404 Not Found if it doesn’t exist
• Operating on the row if it does exist

Cloesce provides a shortcut for this common pattern with instance methods.

An instance method is an API method that calls some Data Source get method to hydrate an instance of a Model, and then passes that instance to the API method implementation.

For example:

model Person for Db {
    primary {
        id: int
    }
}

api Person {
    self get myself -> Person { }
}

The above example is hydrated with Person’s default Data Source get method, which will retrieve the Person instance by its primary key id. If the instance does not exist, a 404 Not Found response will be returned to the client.

It can be implemented in TypeScript like so:

import { Api } from "@cloesce/backend.js";

export const myself: Api.Person.myself = (self) => self;

The self parameter is a flat object containing all of the fields of that Person instance returned by the (default) Data Sources get method.

Using a Custom Data Source

By default, all API methods will use the Default Data Source to hydrate the self instance.

Specify a custom data source like so:

model Person for Db {
    primary {
        id: int
    }

    r2 Bucket::avatars(id) {
        avatar
    }
}

source WithoutAvatar for Person {
    include {
        // Empty!
    }
}

api Person {
    self(WithoutAvatar) get myself -> Person { }
}

In the above code, the myself API method will use the WithoutAvatar data source to hydrate the self instance, which excludes the avatar field.

Any API method can be hydrated with any Data Source (for the same Model).

[internal] Models

A Model may be sensitive and confined to only the backend of your application by using the [internal] tag.

This will prevent any API from accepting that Model as a parameter or returning it as a result, and will not let any public Model compose a relationship with that Model.

No interface will reach the generated client. However, static APIs may be created for that Model, exposing only the methods you want to the client:

[internal]
model UnHashedPassword for Db {
  primary {
    id: int
  }

  column {
    password: string
  }
}

api UnHashedPassword {
  get isThisMyPassword -> bool {
    id: int
    password: string
  }
}

Because one static API method is defined, the client will be able to call UnHashedPassword.isThisMyPassword with an id and password, and receive a boolean result.

The fields of UnHashedPassword will not be exposed to the client.

Execution Context

Durable Objects do not define just an area for storing data, but a single threaded execution context.

Any method may be executed in the context of a Durable Object using Dependency Injection. For example:

durable CounterDo {
    shard {
        tenant: string
    }
}

model Counter {}
api Counter {
  put increment -> int {
    tenant: string

    inject { CounterDo::tenant }
  }
}

Because increment injects an instantiated instance of CounterDo, any code within increment will be executed in the context of that Durable Object, allowing you to safely manipulate data stored in that Durable Object without worrying about race conditions.

Important

Only one instance of a Durable Object can be injected into a method at a time, since each instance represents a single threaded execution context.

Note

If a Durable Object has no shard keys, it is effectively a singleton, and can be injected as:

put method {
   inject { CounterDo::{} }
}

Note

Injecting the Durable Object namespace is different than injecting an instance of that durable object.

For example, inject { CounterDo } would inject the namespace, allowing you to create and manage instances of that Durable Object within your method, but not execute code within the context of any particular instance.

Data Source Execution Context

A Data Source may execute in the context of a Durable Object with the same syntax as an API method.

This means that any API method that uses that Data Source to hydrate self will also execute in the context of that Durable Object.

An instance method already runs inside the Durable Object injected by the Data Source that hydrates its self. Injecting that context again in the API method is a compile error; omit the injection and use the one inherited from the Data Source.

By default, the get method of a Data Source injects its host Model’s Durable Object.

Note

The [instance] tag may only be applied to a parameter that names a column or primary field of the Model. Shard keys and route fields do not qualify, and must be passed as ordinary parameters.

durable CounterDo {
    shard {
        tenant: string
    }
}

model Counter for CounterDo::tenant {
    primary {
        id: int
    }
}

source Default for Counter {
    get {
      [instance]
      id: int

      tenant: string

      inject { CounterDo::tenant }
    }
}

source OutsideContext for Counter {
    get {
      [instance]
      id: int
    }
}

api Counter {
    // Executed inside of CounterDo
    self get myself -> Counter { }

    // Executed outside of CounterDo
    self(OutsideContext) get outside -> Counter { }
}

Streams

If a JSON body is defined in an API method, Cloesce will parse and validate the body before passing it to the method implementation. This is suitable for most use cases, but for certain scenarios such as file uploads or real-time data processing, you may want to handle the request body as a stream.

model File {
    primary {
        id: int
    }
}

api File {
    post upload -> File {
      file: stream
    }
    get download -> stream {
      id: int
    }
}

The above code defines two API methods for the File Model:

• POST /File/upload - Accepts a streaming file upload and returns a File instance
• GET /File/download - Returns a streaming response for downloading a file by its ID

The implementation of the upload method would need to handle the incoming stream appropriately by inspecting the ReadableStream passed in as the file parameter. Similarly, the download method would need to return a stream that can be consumed by the client for downloading the file.

Note

By using stream in a request body, you forgo the ability to have any other parameters in the body, other than the stream itself and HTTP headers.

For example, the following will not compile:

post upload -> File {
  file: stream
  name: string
}

Parameters tagged with [header] are still allowed, since they are not part of the request body.

HttpResult

Any method may return an HttpResult to indicate a success or failure of the API call. Failures may include only a status code and message, while successes may include a status code, data, and headers.

If the result of an API method is not an HttpResult, Cloesce will automatically wrap the result in an HttpResult.ok with a 200 OK status code and a Content-Type of application/json.

Both the backend and frontend utilize the HttpResult type to represent the result of a REST API call. This type encapsulates the success or failure of the API call, along with any relevant data or error information.

The HttpResult type is defined as follows:

export class HttpResult<T = unknown> {
  public constructor(
    public ok: boolean,
    public status: number,
    public headers: Headers,
    public data?: T,
    public message?: string,
    public mediaType?: MediaType,
  ) {}

  /**
   * Return some OK result with the given status, data, and headers.
   */
  static ok<T>(status: number, data?: T, init?: HeadersInit): HttpResult<T>;

  /**
   * Return a failure result with the given status, message, and headers.
   * No body may be attached.
   */
  static fail(status: number, message?: string, init?: HeadersInit): HttpResult<never>;
}

For example, with the following schema:

model Garfield for Db {
    primary {
        id: int
    }
}

api Garfield {
    get byId -> Garfield {
      id: int
    }
}

The implementation of the byId method could return an HttpResult like so:

import { Api } from "@cloesce/backend.js";
import { HttpResult } from "cloesce";

export const byId: Api.Garfield.byId = (id) => {
  const today = new Date();
  const isMonday = today.getDay() === 1;

  if (isMonday) {
    return HttpResult.fail(503, "Garfield hates Mondays");
  }

  return HttpResult.ok(200, { id });
};

CRUD Generation

Creating the same CRUD operations for each Model can be tedious. Cloesce provides a way to automatically generate these operations based on your Model definitions and Data Source configurations.

For every public Data Source defined on a Model, Cloesce will utilize the get, save, and list methods of that Data Source to generate CRUD API endpoints for that Model.

See Data Sources for more information on how to define Data Sources.

Note

Tagging a Model with [crud] tells the compiler that all Data Sources on that Model should be exposed to the client for that particular set of CRUD operations. This is a hint to the compiler, and does not affect the backend.

Cloesce will always have all CRUD methods available to the backend. [crud] is only a hint for the client.

Note

A Model marked as [internal] cannot have [crud] applied to it, since it is not exposed to the client.

Note

The delete operation is not currently supported, but will be added in a future release.

Get

By default, the get operation retrieves a single record by its primary key, shard fields and route fields. For example:

[crud get]
model Person for PersonDo::tenant {
    primary {
        id: int
    }
}

source Custom for Person {
    get {
        special_id: string
    }
}

The above schema will generate two API methods:

• GET /Person/$get: Accepts arguments tenant and id, hydrates with the Default Data Source, and returns a Person instance if a record is found

• GET /Person/$get_Custom: Accepts argument special_id, hydrates with the Custom Data Source, and returns a Person instance if a record is found

List

The list operation retrieves multiple records. By default, it will use a seek based pagination strategy. For example:

[crud list]
model Person for Db {
    primary {
        id: int
    }
}

source OffsetPagination for Person {
    list {
        offset: int
        limit: int
    }
}

The above schema will generate two API methods:

• GET /Person/$list: Accepts arguments limit and lastSeen_id, hydrates with the Default Data Source, and returns a paginated list of Person instances

• GET /Person/$list_OffsetPagination: Accepts arguments offset and limit, hydrates with the Custom Data Source, and returns a paginated list of Person instances

Note

Not all Models truly support list operations. A Model with no SQLite backing will simply return a singleton array, because there is no way to enumerate all instances of that Model.

Save

The save operation creates or updates any record within a Data Source’s include tree.

The only parameter save accepts is a partial Model instance, which is an object that may contain a subset of the Model’s fields. For example:

[crud save]
model Person for Db {
    primary {
        id: int
    }
}

R2 Fields

If your Model contains an R2 field, the save operation will not be able to accept any data for that field, since the ORM is designed only for JSON serializable data.

To work around this, you can define a custom instance method on your Model that accepts a stream parameter:

model Person {
    route {
        id: int
    }

    r2 Bucket::photos(id) {
        avatar
    }
}

api Person {
    self post upload_photo {
        body: stream

        inject { Bucket }
    }
}

Dependency Injection

Any API method may inject Environment Bindings, or inject custom interfaces defined in the schema.

The Cloesce ORM is invoked through dependency injected bindings. To access any generated helper, Data Source, or API method from within an API method, you must inject it at the schema level.

Injecting Environment Bindings

To inject an Environment Binding, add the inject tag to the API method and specify the name of the binding you want to inject:

d1 { Db }

r2 Bucket {
    image {
        id: string
    }
}

var {
    SECRET: string
}

model Person for Db {
    primary {
        id: int
    }
}

api Person {
    get stuff -> Person {
        inject {
            Db
            Bucket
            SECRET
        }
    }
}

A generated backend stub for the stuff API method will include an env parameter with ORM upgraded types, each reached by its declared binding name:

• env.Db will contain all Models within the Db binding, with all of their Data Sources and API methods invokable
• env.Bucket will contain all templated R2 methods for the Bucket binding, with read, write and list methods invokable
• env.SECRET will contain the value of the SECRET binding, as a string

See the Cloesce ORM for more information on how to use the injected bindings.

Defining Custom Inject Bindings

Custom values beyond Worker resources can be defined and injected into API methods.

For example, you may want to create an Auth dependency that any API method can inject to perform authentication and authorization checks:

// Define a custom interface to be injected
inject { Auth }

// Define an API method that injects the Auth interface
api Person {
    get stuff -> string {
        inject { Auth }
    }
}

In order for the dependency to resolve at runtime (any missing dependency is a 500 error), you must register an implementation in the backend:

import { Api, Auth, CfEnv, createApp, Person } from "@cloesce/backend.js";
import { HttpResult } from "cloesce";

// Give the Auth interface a type
declare module "./backend.js" {
  interface Auth {
    username: string | null;
  }
}

const stuff: Api.Person.stuff = (env) => HttpResult.ok(200, `my username is ${env.Auth.username}`);

export default {
  async fetch(request: Request, env: CfEnv): Promise<Response> {
    return createApp()
      .worker(env)
      .register(Auth, { username: "john_doe" })
      .register(Person, { stuff })
      .run(request);
  },
};

Runtime Validation

When an HTTP request is made to the Cloesce Router, incoming data will first be matched to an existing API implementation, and then validated against the schema for that API.

Each type is validated in accordance with the rules defined in the Type Reference. If any validation errors occur, a 400 Bad Request response will be returned with details about the validation errors.

In addition to this, several Validator Tags are also supported for more complex validation scenarios.

Overview

Validator Tags can be applied to any field (i.e. it follows the syntax field: type) in a Model, API parameter, or Data Source parameter.

A foreign key field will automatically inherit all validators from the field it references. For example:

d1 { Db }

model User for Db {
    primary {
        [gt 0]
        id: int
    }
}

model Post for Db {
    primary {
        id: int
    }

    foreign User::id {
        userId
    }
}

In the above code, the userId field in the Post Model will automatically have the [gt 0] validator applied to it, since it is a foreign key referencing the id field in the User Model.

Numerical Validators

These validators apply to the int and real types:

String Validators

These validators apply to the string type:

Cloesce uses Rust’s regex crate for regular expression validation and evaluation. A regex pattern is provided as a regex literal:

[regex /^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/]
email: string

Plain Old Objects

In addition to Models, Cloesce supports Plain Old Objects (POOs) for structured data that doesn’t require database backing, such as data transfer objects (DTOs) or view models.

POOs are defined with the poo keyword and can have fields just like Models, but they lack the ORM and API capabilities that Models have.

Defining a POO

To define a POO, you can use the following syntax:

poo PersonDto {
    id: int
    name: string
    age: int
}

The above code defines a POO called PersonDto with three fields: id, name, and age. You can use this POO in your API definitions, data sources, or anywhere else you need to represent structured data without the overhead of a full Model.

POO Composition

POOs can also be composed of other POOs, allowing you to create complex data structures. For example:

poo GraphNode {
    id: int
    value: string
    children: array<GraphNode>
}

In the above code, the GraphNode POO has a field children which is an array of GraphNodes, allowing you to represent tree-like structures.

[internal] POOs

A Plain Old Object can be marked as [internal], which prevents any API method from accepting or returning that POO. For example:

[internal]
poo UserCredentials {
    password: string
}

kv Credentials {
    creds -> UserCredentials {
        username: string
    }
}

Here, UserCredentials is internal, so no API method can accept or return it. It can still be used in a KV template, since KV templates are not exposed to the client.

A Model may still have a Credentials::creds KV field, but that field cannot be exposed through an API method, since UserCredentials itself is internal.

ORM Reference

Cloesce takes a different approach to the traditional ORM: focus on hydrating data across cloud resources and crud operations, rather than hosting a complex query-builder.

Additionally, unlike other frameworks that combine an ORM with a REST API (such as Django, Rails, or Coalesce), Cloesce does not use an Active Record pattern, deliberately separating generated database types from database persistence.

The Cloesce Environment

Every Cloudflare Workers application defines a set of Environment Bindings. These are available to the application at runtime.

Cloesce upgrades these bindings to provide a rich set of functionality for your application, and exposes them only to methods that explicitly inject them.

Tip

The full set of upgraded bindings is exposed as the env property of the app, once a source has been bound with worker or durable. This allows you to utilize the bindings in your own middleware, or tests.

import { createApp } from "@cloesce/backend.js";

// ...
const app = createApp().worker(env).register(...);
await app.env.Db.Person.get(1);

Note

Every property on the upgraded env is reached by exactly the name it is declared with in the schema (Db, Person, MyKv, …).

KV, R2, and Durable Object Methods

Key templates can be defined in R2, Durable Object KV, and Durable Object bindings.

Each upgraded binding exposes methods to read, write, and list data from these key templates.

durable MyDo {
    shard {
        tenant: string
    }

    settings -> json {
        id: string
        "custom/key/template/{id}"
    }
}

r2 MyBucket {
    image {}
}

kv MyKv {
    user -> json {
        id: int
        "user/{id}"
    }
}


// ...
api Person {
    get method {
        tenant: string

        inject {
            MyDo
            MyDo::tenant
            MyBucket
            MyKv
        }
    }
}

KV

interface KvHelpers<T> {
  /** Renders the key template. */
  template(id: number): string;

  /** Reads the value at the templated key. */
  get(id: number): Promise<T | null>;

  /** Writes the value at the templated key. */
  put(id: number, value: T): Promise<void>;

  /** Lists keys under this template's prefix. */
  list(options?: {
    limit?: number;
    cursor?: string;
  }): Promise<{ keys: { key: number; value: T }[]; cursor?: string }>;
}

Example Usage

env.MyKv.user.template(1); // => "user/1"
await env.MyKv.user.put(1, { hello: "world" });
await env.MyKv.user.get(1); // => { hello: "world" } | null
await env.MyKv.user.list({ limit: 20 });

R2

interface R2Helpers {
  /** Renders the key template. */
  template(): string;

  /** Reads the object's data. */
  get(): Promise<R2ObjectBody | null>;

  /** Writes the object's data. */
  put(
    value: ReadableStream | ArrayBuffer | ArrayBufferView | string | Blob,
  ): Promise<R2Object | null>;

  /** Lists object HEADs (not their data) under this template's prefix. */
  list(options?: {
    limit?: number;
    cursor?: string;
    delimiter?: string;
  }): Promise<{ objects: R2Object[]; cursor?: string }>;
}

Example Usage

env.MyBucket.image.template(); // => "image"
await env.MyBucket.image.put(new Uint8Array([1, 2, 3]));
await env.MyBucket.image.get(); // => R2ObjectBody | null
await env.MyBucket.image.list({ limit: 20 });

Durable Object

interface DoHelpers<T> {
  /** Resolves the shard's DurableObjectId. */
  id(tenant: string): DurableObjectId;

  /** Resolves a stub for the shard, typed as `T`. */
  stub<T>(tenant: string): DurableObjectStub & T;
}

Example Usage

env.MyDo.id("tenant");
env.MyDo.stub<MyDo>("tenant");

Durable Object KV

interface DoKvHelpers<T> {
  /** Renders the key template. */
  template(tenant: string): string;

  /**
   * Reads the value at the templated key.
   * Pass shard fields to call from outside the DO (async, routed over RPC).
   * Pass a `DurableObjectState` to call from inside the DO's own methods
   * (synchronous, reads local storage directly).
   */
  get(tenant: string): Promise<T | null>;
  get(ctx: DurableObjectState): T | null | undefined;

  /** Writes the value at the templated key. Same overload as `get`. */
  put(tenant: string, value: T): Promise<void>;
  put(ctx: DurableObjectState, value: T): void;

  /** Lists keys under this template's prefix. */
  list(ctx: DurableObjectState): { key: string; value: T }[];
}

Example Usage

env.MyDo.settings.template("tenant"); // => "custom/key/template/tenant"
await env.MyDo.settings.put("tenant", { hello: "world" }); // outside the DO
await env.MyDo.settings.get("tenant"); // outside the DO

Model Methods

When a D1 or Durable Object database is injected into an API method, all Models defined against that database become available in their upgraded form, as ModelStore objects.

Every method on a ModelStore (get, list, save, hydrate, hydrateAll, load) returns an HttpResult<T>. This is the same result wrapper your API methods return:

const result = await env.Db.Person.get(1);
if (!result.ok) return result; // propagate the 404/400
const person = result.data!;

Default and Named Data Sources

Every ModelStore exposes the model’s Default Data Source directly as get, list, and save:

await env.Db.Person.get(1); // Promise<HttpResult<Person>>
await env.Db.Person.list(0, 20); // Promise<HttpResult<Person[]>>
await env.Db.Person.save({ id: 1, name: "Ada", age: 30 }); // Promise<HttpResult<Person>>

Every other Data Source is available as its own property, keyed by its declared name. It exposes the same get/list/save shape, scoped to that source’s include tree and parameters:

await env.Db.Person.OverAge.list(18, 0, 20); // scoped to the `OverAge` source

hydrate and hydrateAll

Both the Default Data Source and every named Data Source expose hydrate and hydrateAll. These turn partial, already-fetched rows into fully hydrated Models according to that source’s include tree.

• hydrate(row): takes one partially-loaded row, mutated in place and consumed. Fills in whatever relations aren’t already present on it.
• hydrateAll(rows): same, but for an array of rows. The array becomes the complete root set, so no root fetch is issued. Only the missing relations are fetched.

Tip

Hydration is guided by what’s already on the row. A field or relation that’s already present (even as []) is treated as authoritative and skipped.

This lets you interleave hand-written queries with the ORM. Write your own SQL for a custom filter, then hand the raw rows to hydrateAll to fill in everything else: R2 fields, KV fields, related Models across other bindings.

r2 Avatars {
    avatar {
        pId: int
    }
}

model Person for Db {
    primary {
        id: int
    }

    column {
        name: string
        age: int
    }

    r2 Avatars::avatar(id) {
        avatar
    }
}

source OverAge for Person {
    include {
        avatar
    }

    list {
        age: int
        lastId: int
        limit: int

        inject { Db }
    }
}

To implement the OverAge::list method, you could do the following:

import { Api } from "@cloesce/backend.js";

export const overAge = {
  async list(env, age, lastId, limit) {
    // Get the list of people over 18 from the database
    const res = await env.Db.prepare(
      `SELECT * FROM Person WHERE age > ?1 AND id > ?2 ORDER BY id ASC LIMIT ?3`,
    )
      .bind(age, lastId, limit)
      .all();

    // Ta-da! Cloesce turned all of the database row results into fully hydrated Person
    // instances, with their R2 fields populated.
    return env.Db.Person.OverAge.hydrateAll(res.results);
  },
} satisfies Api.Person.OverAge.Of;

load

Unique to the ModelStore itself (not on named Data Sources) is load. It takes a Model value you already hold and an ad-hoc IncludeTree, and returns a hydrated copy without consuming or mutating the original.

Where hydrate/hydrateAll run a source’s precompiled include tree, load plans its tree at runtime. You can shape it per call:

async feed(self, env) {
  const full = await env.SubRedditDb.SubReddit.load(self, {
    posts: {
      post: {
        meta: {},
        comments: {},
      },
    },
  });
  return full.data?.posts.map((p) => p.post) ?? [];
}

load is the escape hatch for cases a compile-time Data Source doesn’t cover, like hydrating different relations depending on a runtime flag.

Cloesce Query Planner

How the heck does Cloesce know how to hydrate a Model, with data that could be stored anywhere?

Cloesce splits query work into two parts:

• The Query Planner looks at a Model’s schema and an include tree, and decides what to fetch, from where, and in what order.
• The Query Executor walks that plan at runtime, and issues the reads and writes.

Plans are of two different IR forms: select and save.

Select Plans

Select plans are either generated at compile time (for every Data Source), or can be dynamically made at runtime (for the load method). The runtime path is a call to a WASM module that implements the planner.

Plans consist of stages: a sequence of operations that must be performed in order, blocking on each stage until the next stage can be executed.

Every stage consists of one or more steps: a single operation that can be executed in parallel with other steps in the same stage.

The planner’s goal is to minimize the number of stages, and maximize the number of steps in each stage, such that as much work as possible can be done in parallel.

Note

Nested many relationships are batch-loaded such that the N+1 query problem is mitigated.

Save Plans

Save plans work in the other direction. Given a partial payload to save, the planner figures out:

• What to INSERT/UPDATE, and where.
• In what order writes must happen, so foreign keys and shard fields resolve correctly.
• What needs to be read back afterward (e.g. an autoincrementing primary key) before a dependent write can use it.

Save plans can’t be fully precompiled. The shape of what’s being saved depends on which fields are present in a given call’s payload, so they’re planned once per call, at runtime.

Writes to the same backend that don’t depend on each other’s readback values are grouped into a single batch. Writes that need a value produced by an earlier stage, like a just-inserted parent ID referenced as saved.board.pid, wait for the stage that produces it.

Explain Command

The cloesce explain CLI command prints the exact plan the planner produced for a given Model, Data Source, and operation.

The same text is embedded as the @remarks block on the generated method’s JSDoc. It also shows up in your editor’s hover tooltip.

cloesce explain <model> <data_source> <get|list|save> [--dir .] [--payload <file.json>]

• get/list print the precompiled plan directly, no payload needed.
• save requires --payload <file.json>, a JSON body shaped like what you’d pass to that source’s save method, since the plan depends on the payload’s shape.

Each step in the printed tree is one operation. Here’s what the grammar means:

cloesce explain Org Default get prints:

SELECT PLAN (GET) `Org` · 3 stages · 5 steps
INCLUDE
└─ `board`
   ├─ `banner`
   ├─ `entries`
   └─ `top`

STAGE 0
└─ SEARCH `Org` ON d1 `db` ONE

STAGE 1
├─ SCAN `Board` ON durable `BoardDo` INTO `board` ONE
│     JOIN `parent.tenantId` = `row.tenantId`
│     SHARD `tenantId` = tenantId
│     ATTACH `tenantId` = tenantId
└─ READ durable `BoardDo` KEY "top" INTO `board.top` SHARD `tenantId` = tenantId

STAGE 2
├─ READ r2 `Bucket` KEY "banners/{board.pid}" INTO `board.banner`
└─ SEARCH `Entry` ON durable `BoardDo` INTO `board.entries` MANY
      JOIN `parent.tenantId` = `row.tenantId` AND `parent.pid` = `row.boardId`
      SHARD `tenantId` = tenantId
      ATTACH `tenantId` = tenantId

cloesce explain Org Default save --payload payload.json prints the save plan. The payload here creates an Org along with its Board, one Entry, and a KV-backed top:

SAVE PLAN `Org` · 1 stage · 3 steps
INCLUDE
└─ `board`
   ├─ `banner`
   ├─ `entries`
   └─ `top`

STAGE 0
├─ BATCH ON d1 `db`
│  ├─ INSERT `Org` (`tenantId` = 7)
│  └─ READBACK `Org` INTO `result`
├─ BATCH ON durable `BoardDo` SHARD `tenantId` = 7
│  ├─ INSERT `Board` DEFAULT VALUES
│  ├─ INSERT `Entry` (`score` = 42, `boardId` = `saved.board.pid`)
│  ├─ READBACK `Board` INTO `board`
│  └─ READBACK `Entry` INTO `board.entries[0]`
└─ WRITE durable `BoardDo` KEY "top" INTO `board.top` SHARD `tenantId` = 7
   └─ VALUE {"cached":true}

Note

Entry’s insert shares a stage with Board’s insert-and-readback because it’s batched onto the same Durable Object round trip.

banner appears in the include tree but produces no step: R2 fields are read-only, so a save never writes to a bucket.

Warning

save does not write R2 objects. An R2 field hydrates as an R2Object HEAD metadata such as key, size, and etag, not the object’s bytes, so there is no value a save payload could carry that would meaningfully round-trip into the bucket. Including an R2 field in a save payload is silently ignored.

Upload through the bucket binding directly:

await env.Avatars.avatar.put(user.id, bytes);

Reads are unaffected: an included R2 field still hydrates on get, list, and hydrateAll.