The Cloesce Schema Language

Note

Cloesce is under active development, expanding its feature set as it pushes toward full Cloudflare support across any language.

The syntax and features described here are subject to change as the project evolves.

Cloesce is a schema language that describes a full stack application built on Cloudflare’s edge ecosystem.

Contributing

Contributions are welcome at all levels. Join our Discord to discuss ideas, report issues, or get help getting started. Create an issue on GitHub if you find a bug or have a feature request.

Coalesce

Check out Coalesce, an accelerated web app framework for Vue.js and Entity Framework by IntelliTect. Many core concepts of Cloesce come directly from Coalesce (Cloesce = Cloudflare + Coalesce).

Resources

LLMs

Interact with this documentation with an LLM by utilizing the llms-full.txt found here. Download from the terminal using curl:

curl https://cloesce.pages.dev/llms-full.txt -o llms-full.txt

VS Code Extension

A basic language highlighting extension for Cloesce is available in the VS Code marketplace. In the future, this extension will also include a full LSP server.

Getting Started

Tip

Cloesce runs on Cloudflare Workers. Familiarity with Workers and Wrangler is recommended. If you are new to Workers, check out the official Cloudflare Workers documentation.

Welcome to the Getting Started guide. This guide covers:

• Installing Cloesce
• A basic project structure with create-cloesce
• Building, migrating, and running your application

Installation

Note

Only TypeScript compilation is supported.

Support for additional languages will be added in future releases.

Installing the Compiler

Linux and macOS

curl -fsSL https://cloesce.pages.dev/install.sh | sh

Windows (PowerShell)

irm https://cloesce.pages.dev/install.ps1 | iex

Then verify the installation:

cloesce version

Starting a New Project

The fastest way to get a Cloesce project up and running is to use the create-cloesce template.

Prerequisites

1. Sign up for a Cloudflare account (not necessary for local development)
2. Install Node.js (version 16.17.0 or later)

create-cloesce

Run the following command in your terminal:

npx create-cloesce my-cloesce-app

After running the command, navigate into your new project directory:

cd my-cloesce-app

A simple project structure is created for you.

├── src/
│   ├── api/            # API route handlers
│   ├── web/            # Frontend web assets
│   └── schema/
│       └── schema.clo  # Cloesce schema definition
├── test/               # Unit tests for example Models
├── migrations/         # Database migration files
├── cloesce.jsonc       # Cloesce configuration
└── package.json

Building and Migrating

Configuration

Define a cloesce.jsonc file in your project root to configure the Cloesce compiler:

{
  "src_paths": ["./src/schema"],
  "workers_url": "http://localhost:5000/api",
  "wrangler_config_format": "jsonc" // or "toml"
}

Tip

Multiple configuration files can be defined for different environments (e.g., staging.cloesce.jsonc, production.cloesce.jsonc).

Select the desired configuration file using the --env flag when running Cloesce commands:

cloesce --env staging ...

Compilation

Compilation will transform your Cloesce schema into backend stubs and a client side API under the .cloesce directory. In your root directory, run the following command to compile your schema:

cloesce compile

Important

Any generated artifacts should not be modified directly or committed to source control. Simply import them into your backend and client code, relying on a build step to run the Cloesce compiler and keep the generated code up to date.

Migrations

Tip

Schema modifications to a SQLite backed Model should be accompanied by a new migration. This ensures that your database schema stays in sync with your Cloesce Models.

Cloesce supports any number of SQLite databases in a single project. To generate SQL migration files for a specific binding, run the following command:

cloesce migrate --binding <binding> <migration-name>

To generate migrations for all bindings in your project, use the --all flag:

cloesce migrate --all <migration-name>

Apply D1 Migrations

Cloesce generate the SQL for migrations, but not apply them,

If a D1 database is being utilized, you must apply the generated migrations using the Wrangler CLI:

npx wrangler d1 migrations apply <binding-name>

Running

After compilation and migrations, run your application locally with Wrangler:

npx wrangler dev --port <port-number>

Deploying

Deploy your application to Cloudflare’s edge with Wrangler:

npx wrangler deploy

Type Reference

This section provides a reference for the types available in the Cloesce Schema Language. These types can be used to define your application’s data Models, APIs, Data Sources, and more.

All Types

Primitives

Generics

Objects

Any Model or Plain Old Object defined in your schema can be referenced as a type. For example, to have a Plain Old Object that references a Model:

model User for Db {
    primary {
        id: int
    }

    column {
        name: string
    }
}

poo Profile {
    user: User
    bio: string
}

SQLite Compatible Types

By default, all of these types are NOT NULL in a SQLite database.

To allow NULL values, wrap the type in the option generic, e.g. option<string>.

Environment Declaration

Environment bindings are an easy way to declare, manage, reference and inject Cloudflare Workers bindings across your application.

Currently, Cloesce supports D1, KV, R2, Durable Objects, and custom Wrangler Environment Variables.

Tip

Any top level declaration in Cloesce is global across any file in the project. This means that environment bindings declared in one file can be referenced and used in any other file.

Tip

Environment bindings can be injected to any API implementation via the [inject] tag.

Workers KV and R2

Tip

KV and R2 binding templates can be referenced on any Model via KV Fields and R2 Fields, allowing you to easily integrate Workers KV and R2 across the full stack of the application.

Note

The Cloesce Compiler will not allow a binding template key format to collide with any other template in the binding.

For example, if some binding were to have a key foo/{bar}, then no template would be allowed to have the leading prefix foo/. This allows the list prefix matching functionality of Workers KV and R2 to work without ambiguity.

Note

KV definitions in the schema do not yet support cache control directives and expiration times. This is planned for a future release.

Workers KV

Cloudflare KV is a globally distributed key-value store. Cloesce provides first class support for KV, allowing a simple binding declaration to generate not only a Wrangler configuration for the namespace, but also a fully typed interface for querying that namespace in your application code.

Cloudflare KV has a simple toolset and API for storing and retrieving key-value pairs. Specifically, KV supports:

• Eventually consistent writes
• List queries with key based prefix matching and pagination
• Metadata on each key-value pair
• Maximum 25MB value size limit

To define a Workers KV namespace binding, use the kv block:

kv MyNamespace {
    // ... define as many binding templates as necessary
    settings() -> json {
        "path/to/settings"
    }

    // accept any number of parameters and use them
    // in the template string to generate dynamic keys
    session(token: string) -> SessionToken {
        "sessions/{token}"
    }
}

In the above example MyNamespace is the name of a KV namespace, and settings and session are “binding templates” defined on that namespace.

Every binding template describes a location of data in the KV namespace, and the return type of the template describes the expected type of that data. The template string can be generated from any number of parameters, allowing for dynamic keys.

This definition will compile to an interface capable of querying the KV namespace with the defined key templates. For example:

settings: {
  template: () => `path/to/settings`,

  get: () => namespace.get(`path/to/settings`),

  put: (value) => namespace.put(`path/to/settings`, value),

  list: (options: { limit, cursor}) =>
    namespace.list({ ...options, prefix: `path/to/settings` }),
},

session: {
  template: (token) => `sessions/${token}`,

  get: (token) => namespace.get(`sessions/${token}`),

  put: (token, value) =>
    namespace.put(`sessions/${token}`, value),

  list: (options: { limit, cursor }) =>
    namespace.list({ ...options, prefix: `sessions/` }),
},

// ...

These methods will be merged on top of the Cloudflare KVNamespace interface in the Cloudflare Environment.

In addition to the backend interface, a Wrangler configuration will be generated:

[[kv_namespaces]]
binding = "MyNamespace"
id = "replace_with_my_namespace_id"

R2

Note

R2 is used to store large unstructured data. For this reason, Cloesce will not query and buffer the full value of an R2 field into the Worker runtime.

Instead, only a HEAD request is made to R2 to check for existence and retrieve metadata.

To define a Cloudflare R2 bucket binding, use the r2 block:

r2 MyBucket {
    // ... define as many binding templates as necessary
    getObject(key: string) {
        `path/to/${key}`
    }
}

Unlike Workers KV, R2 bindings do not have a return type, because they will always return the Cloudflare R2Object (a HEAD request to the object, not the full value).

A similar interface to the above MyBucket definition will be generated for querying the R2 bucket and accessing objects with the defined key templates.

In addition to the backend interface, a Wrangler configuration will be generated:

[[r2_buckets]]
binding = "MyBucket"
bucket_name = "replace-with-my_bucket-name"

D1

Cloudflare D1 is a distributed SQL database built on SQLite for Workers. Cloesce allows any number of D1 databases to be defined in the schema, and allows Models to represent tables in those databases.

Tip

D1 is limited to 10GB of storage, so it is best suited for the primary database of a small application, or for a queryable cache layer in front of a larger database.

For more complex applications, consider using Durable Objects, which can provide virtually unlimited storage across shards with SQLite.

Defining a D1 Binding

To define a D1 environment binding, use the d1 block within the env block:

d1 {
    MyDb
}

Unlike Workers KV and R2, no binding templates can be defined on a D1 binding (though, in the future, compiler verified SQL query templates will be supported!).

To interact with the D1 database, define Models backed by the binding:

model User for MyDb {
    primary {
        id: int
    }

    column {
        name: string
    }
}

See more on D1 Backed Models in the Models chapter.

Wrangler Configuration

A Wrangler configuration will be generated for each D1 binding defined in the schema:

[[d1_databases]]
binding = "MyDb"
database_id = "replace_with_mydb_id"
database_name = "replace_with_mydb_name"
migrations_dir = "migrations/MyDb"

Migrations

To generate SQL migration files for a specific binding, run the following command:

cloesce migrate --binding <binding> <migration-name>

Apply the generated migrations for a D1 database using the Wrangler CLI:

npx wrangler d1 migrations apply <binding-name>

Durable Objects

Cloudflare Durable Objects provide a way to run stateful code on Cloudflare’s edge network.

To describe them simply (a task which is difficult to do justice), Durable Objects are:

1. A place to store data (SQLite and KV storage).
2. A single threaded sequential execution context.
3. Capable of being sharded across any number of instances (think database-per-X).

Cloesce provides first class support for Durable Objects, allowing you to define them in your schema, generate a fully typed interface, injecting execution context into your API implementations, and using them as a Model backing.

Tip

Durable Objects are not a Model, but rather a place that any number of Models can be backed by.

A Durable Object instance can store any number of Models within its SQLite and KV storage, and can execute any code necessary to manage those Models.

For more on using Durable Objects as a Model backing, see the Models chapter.

Warning

Cloesce is only capable of using the modern SQLite backed Durable Objects, and does not support the legacy Durable Object storage API.

Defining a Durable Object Binding

To define a Durable Object environment binding, use the durable block:

durable MyShardedDo {
    shard {
        tenant: int
    }

    // ... define as many binding templates as necessary
    settings() -> json {
        "settings"
    }

    userMap(userId: int) -> json {
        "user/{userId}"
    }
}

durable MyGlobalDo {
    settings() -> json {
        "settings"
    }
}

The above example defines two Durable Object bindings:

• MyShardedDo: A sharded Durable Object, meaning any number of Durable Object instances can be created with different shard parameters. In this case, the tenant parameter is used to shard the Durable Object by tenant ID.

• MyGlobalDo: A global Durable Object, meaning Cloesce will treat it as if there is only one instance of the Durable Object, and will not allow any shard parameters to be defined.

In both bindings, KV templates can be defined to generate a typed interface for interacting with the Durable Object’s KV storage.

In the case of MyShardedDo, the userMap template will generate an interface for storing user data in the Durable Object’s KV storage, with keys formatted as "user/{userId}".

Generated Interface

Cloesce will create an abstract class to extend for each Durable Object binding defined in the schema, along with helper functions merged into the Cloesce Env type.

The generated abstract class provides:

• KV template accessors (e.g. this.settings, this.userMap(userId)) with get, put, list, and template methods for interacting with the Durable Object’s KV storage.

• A cloesce method for applying generated migrations as well as invoking the Cloesce Router.

The generated Env helpers provide:

• env.MyShardedDo.template(tenant) — returns the shard key string for a given set of shard parameters.
• env.MyShardedDo.id(tenant) — returns a DurableObjectId for the given shard parameters.
• env.MyShardedDo.stub(tenant) — returns a typed DurableObjectStub for the given shard parameters.

For global Durable Objects (no shard fields), these helpers take no arguments.

Extending the Durable Object Class

The Cloesce Router will forward HTTP requests bound for a particular Durable Object from the Worker to the fetch method of the generated Durable Object class.

To implement custom logic for handling these requests, extend the generated Durable Object class and implement the fetch method:

import * as clo from "@cloesce/backend.js";

export class MyShardedDo extends clo.MyShardedDo {
    app: CloesceApp;

    constructor(state: DurableObjectState, env: clo.CfEnv) {
        super(state, env);
        this.app = this.cloesce(env, [...migrations]);
        this.app.register(...);
    }

    async fetch(request: Request): Promise<Response> {
        return await this.app.run(request);
    }
}

Here, the MyShardedDo class extends the generated clo.MyShardedDo class, and implements the fetch method to handle incoming HTTP requests.

The cloesce method is used to create a Cloesce application instance, which can be used to register API implementations and run the application.

Wrangler Configuration

A Wrangler configuration will be generated for each Durable Object binding defined in the schema:

[[durable_objects.bindings]]
class_name = "MyShardedDo"
name = "MyShardedDo"

[[durable_objects.bindings]]
class_name = "MyGlobalDo"
name = "MyGlobalDo"

[[migrations]]
new_sqlite_classes = [
    "MyShardedDo",
    "MyGlobalDo",
]
tag = "v1"

Backing Models

A Model may be backed by a Durable Object, meaning the data for that Model can be pulled from an instance of a Durable Object. Declarations must explicitly state each shard field on the Model (aliased to any name in shard field order):

durable MyShardedDo {
    shard {
        tenant: int
    }
}

model Tenant for MyShardedDo(tenant) {
    // ...
}

// can alias `tenant` to anything
model Tenant for MyShardedDo(aliasWhatever) {
    // ...
}

In this case, the Tenant Model is backed by the MyShardedDo Durable Object, and the tenant shard field is declared on the Model.

Read more about backing Models with Durable Objects in the Models chapter.

Execution Context

Durable Objects provide a single threaded execution context in which their internal storage may be accessed and mutated.

Just because a Model is backed by a Durable Object does not mean that all API methods and Data Sources that interact with that Model are executed within the Durable Object’s execution context.

Explicitly inject the Durable Object’s execution context into any API method or Data Source method to interact with the Durable Object’s internal storage. For example:

api AnyModel {
    [inject MyShardedDo(tenant)]
    get doSomething(tenant: int) -> json
}

In order to instantiate the MyShardedDo execution context, the tenant shard parameter must be passed in as an argument to the API method, and the method must be decorated with the inject tag.

Read more about execution context injection in the API chapter and the Data Source chapter.

Environment Variables

Any number of environment variables can be defined in the schema, which Cloesce will ensure are placed in the Wrangler configuration and made available to the Worker at runtime.

Defining Environment Variables

Note

Variables are restricted to the same set of primitive types as SQLite Types.

To define an environment variable, use the var block in the schema:

vars {
    MY_VAR: string
    MY_OTHER_VAR: int
}

Inject them into an API endpoint like so:

api Foo {
    [inject MY_VAR]
    get foo() -> string
}

Models

A Model in Cloesce defines a structure hydrated from some sources of persistent data, such as a D1 database, Durable Object, KV namespace, or R2 bucket.

Once defined, a Model is a first class citizen across the frontend, backend, and database layers of your application, capable of housing API endpoints and being serialized for the client.

In this chapter, we will explore the various features of Cloesce Models, including:

• SQLite Backed Models
• SQLite Column Constraints
• KV Fields
• R2 Fields
• Navigation Fields

SQLite Backed Models

Models are able to pull from various sources of data, including SQLite databases stored in D1 and Durable Objects.

When backed by a SQLite database, a Model’s properties will translate to a table in that particular database.

Defining an Environment Binding

To back a Model with a SQLite database, you first need some storage binding that supports SQLite. Two options exist:

// 1. Cloudflare D1
d1 {
    MyDb
}

// 2. Durable Objects
durable MyDurableObject {
    shard {
        tenant: string
    }
}

See more information on D1 and Durable Objects definitions in the Environment chapter.

Defining a Model

With D1

d1 {
    MyDb
}

model User for MyDb {
    primary {
        id: int
    }

    column {
        name: string
    }
}

The above code defines a Model “User” stored in the D1 database MyDb, with several properties:

With Durable Objects

durable MyDurableObject {
    shard {
        tenant: string
    }
}

model User for MyDurableObject(tenant) {
    primary {
        id: int
    }

    column {
        name: string
    }
}

The above code defines a Model “User” stored in the Durable Object MyDurableObject, with several properties:

Across the Stack

Once defined, the User Model is a first class citizen across the frontend, backend, and database layers of your application.

For example, the frontend of your application will generate the following TypeScript type for the User Model:

// .cloesce/client.ts
export class User {
  id: number;
  name: string;

  // iff backed by a Durable Object
  tenant: string;
}

In SQLite, the User Model will be represented as a table:

CREATE TABLE User (
    id INTEGER PRIMARY KEY,
    name TEXT
);

SQLite Column Constraints

Tip

All fields of a SQLite backed Model must be a SQLite compatible type.

This chapter provides a reference for the SQLite specific features of Models.

Primary Key

A primary block is required in every SQLite backed Model. It directly translates to the SQLite PRIMARY KEY constraint.

model User for Db {
    primary {
        id: int
    }
}

Note that by default, primary key fields are NOT NULL, UNIQUE, and AUTOINCREMENT (for integer fields).

Composite Primary Key

Any number of fields can be in the primary block, or, any number of priamry blocks can exist, defining a composite primary key.

For example, the following User Model has a composite primary key consisting of an id field and an email field:

model User for Db {
    primary {
        id: int
        email: string
    }
}

// Equivalent to:
model User for Db {
    primary {
        id: int
    }

    primary {
        email: string
    }
}

Foreign Key

The foreign block allows you to define foreign key relationships between Models. It translates to the SQLite FOREIGN KEY constraint.

model Dog for Db {
    primary {
        id: int
    }
}

model Person for Db {
    primary {
        id: int
    }

    // Person has a foreign key relationship to Dog's field `id`
    // through its own field `dogId`.
    foreign Dog::id {
        dogId
    }
}

Foreign key fields inherit the type of the field they reference. In the above example, Person::dogId is of type int because it references Dog::id, which is of type int. Foreign key fields are also NOT NULL by default, but they do not have to be unique.

Optional Foreign Key

To allow NULL values in a foreign key field, use the optional modifier:

model Person for Db {
    primary {
        id: int
    }

    foreign Dog::id optional {
        dogId
    }
}

Composite Foreign Key

A Model can have a composite primary key by listing multiple fields in a primary block.

Similarly, a Model can have a composite foreign key by listing multiple fields in a foreign block.

model Person for Db {
    primary {
        firstName: string
        lastName: string
    }
}

model Dog for Db {
    primary {
        id: int
    }

    foreign (Person::firstName, Person::lastName) {
        ownerFirstName
        ownerLastName
    }
}

Foreign Primary Key

A field can be both a primary key and a foreign key at the same time. This is useful for representing many-to-many relationships:

model Enrollment for Db {
    primary {
        foreign Student::id {
            studentId
        }

        foreign Course::id {
            courseId
        }
    }
}

model Student for Db {
    primary {
        id: int
    }
}

model Course for Db {
    primary {
        id: int
    }
}

Unique Constraint

The unique (field1, field2, ...) declaration adds a unique constraint over one or more existing fields on a Model. It translates to the SQLite UNIQUE constraint. A field may participate in any number of unique constraints.

model User for Db {
    primary {
        id: int
    }

    column {
        email: string
        username: string
    }

    foreign Profile::id {
        profileId
    }

    foreign Dog::id {
        dogId
    }

    // The combination (email, profileId, dogId) must be unique.
    unique (email, profileId, dogId)

    // `username` must be unique on its own.
    unique (username)

    // `dogId` must also be unique on its own
    unique (dogId)
}

KV Fields

Cloudflare KV is a globally distributed key-value store. Cloesce provides first class support for KV, allowing you to define Models with KV-hydrated fields that seamlessly integrate with D1 backed Models and R2 fields.

Defining an Environment Binding

To use KV fields in your Models, you first need to define an environment binding for the KV namespace in your Cloesce schema:

kv MyNamespace {
    // ...templates
    settings() -> json {
        "settings"
    }

    profile(userId: string) -> json {
        "profile/{userId}"
    }
}

Additionally, a Durable Object binding can also act as a KV namespace:

durable MyDurableObject {
    settings() -> json {
        "settings"
    }

    profile(userId: string) -> json {
        "profile/{userId}"
    }
}

Note

Durable Object storage is only accessible from within the context of that Durable Object.

If want to use a Durable Object as a KV namespace for some Model, that Model must be backed by that Durable Object using for, e.g. model User for MyDurableObject.

You will not be able to use a Durable Object as a KV namespace for a Model that is backed by D1 or has no backing at all.

Read more about KV bindings and Durable Object bindings in the Environment chapter.

Defining a KV Field

Note

Models are not storage-monolithic. A Model that uses SQLite fields can also have KV fields, and R2 fields, all at the same time.

A field in a Model can be hydrated from KV by referencing a binding defined on a kv namespace:

kv MyNamespace {
    settings() -> json {
        "settings"
    }
}

model User {
    kv MyNamespace::settings() {
        settings
    }
}

The above snippet defines a Model User with a KV field settings that is stored in the namespace MyNamespace under the static key “settings”.

settings is typed as json, and Cloesce will automatically handle the serialization and deserialization of this field when reading from and writing to KV.

Quirks of a non-SQLite backed Model

Unlike a SQLite backed Model, the above User Model does not define a backing database with for.

The settings field is therefore not associated with any particular row, and the Model has no underlying “backing”: Cloesce simply hydrates settings from KV whenever you query for a User.

Note

If you query for a User and the settings key does not exist in KV, Cloesce returns null for the field rather than a 404 error (as it would for a SQLite backed Model with a missing row). Cloesce cannot distinguish an absent key from a key that exists with a null value.

Key Interpolation

A common pattern is to format a key such that any number of related values can be stored under that template. For example:

kv MyNamespace {
    profile(userId: string) -> json {
        "profile/{userId}"
    }
}

Here, profile accepts one parameter userId, which is a string. The key for this field in KV is defined as "profile/{userId}", where {userId} is a placeholder that will be replaced with the actual value of the userId parameter when accessing KV.

There are two ways to pass the value for userId when querying for this field:

Route Fields

Note

Route fields and SQLite columns are mutually exclusive. If a Model defines any SQLite columns, it cannot use route fields, and vice versa.

Notably, a Durable Object backed Model can use route fields, iff it does not define any SQLite columns. However, a D1 backed Model cannot use route fields at all, as D1 backing requires the presence of SQLite columns.

Note

A Model that is not backed by any database is commonly referred to as “Worker Backed”, because its data is not persisted in any database and only exists in memory during the execution of a Worker.

Note

route fields are limited to SQLite compatible types

If a Model aims to exist without any SQLite backing at all, it may use route fields to populate the parameters for its KV fields:

model User {
    route {
        userId: string
    }

    kv MyNamespace::profile(userId) {
        profile
    }
}

The route block defines a userId field that is populated from the URL route when querying for a User. For example, if you query for /users/123, Cloesce will populate userId with the value “123”, and then use that value to construct the key for the KV field profile, finally fetching the value from KV.

Columns

If a Model is SQLite backed, it can use the values of its columns to populate the parameters for its KV fields:

model User for MyDb {
    primary {
        id: int
    }

    kv MyNamespace::profile(id) {
        profile
    }
}

The Cloesce ORM will know to first hydrate the table User to get the value of id, and then use that value to construct the key for the KV field profile, finally fetching the value from KV.

Key Template Conventions

KV is capable of being queried by a prefix, listing all keys that exist under it. Cloesce will enforce that the schema does not overlap keys in a way that would make results ambiguous. For example, the following schema would be invalid:

kv MyNamespace {
    profile(userId: string) -> json {
        "profile/{userId}"
    }

    favoriteNumber(userId: string) -> int {
        "profile/{userId}/favNum"
    }
}

In this example, the template

• "profile/{userId}/favNum"

overlaps with

• "profile/{userId}"

A prefix list on "profile/" would include "profile/{userId}/favNum" because it matches the prefix, even though it does not conform to the profile template.

Cloesce throws an error when validating this schema, preventing such ambiguities.

Generated Code

KValue

When pulling from a KV namespace (not Durable Object storage), Cloesce will return an instance of the KValue class for that field:

// both .cloesce/client.ts and .cloesce/backend.ts
export class KValue<V> {
  raw: unknown | null;
  metadata: unknown | null;

  get value(): V | null {
    return this.raw as V | null;
  }
}

Cloesce will make no effort to validate that the raw value actually conforms to the expected type V (aside from validating request parameters).

It is up to you to ensure that the data stored in KV is of the correct shape, and to handle any cases where it is not.

Backend Helpers

For each KV template and Durable Object template, Cloesce will generate accessor methods to get, put, and list keys in the corresponding KV namespace.

For example, for the schema:

kv MyNamespace {
    settings() -> json {
        "settings"
    }

    profile(userId: string) -> json {
        "profile/{userId}"
    }
}

Cloesce will merge the KVNamespace or DurableObject interfaces with the following generated methods:

where env is the Cloesce Environment.

R2 Fields

Cloudflare R2 is a globally distributed object storage service that allows you to store and serve large amounts of unstructured data, such as images, videos, and other media files. With Cloesce, you can easily integrate R2 into your application by defining R2 fields in your Models.

Many of the same concepts and syntax for defining KV fields in Cloesce also apply to R2 fields. Read the KV fields chapter for more information.

Defining an Environment Binding

To use R2 fields in your Models, you first need to define an environment binding for the R2 bucket in your Cloesce schema:

r2 MyBucket {
    // ...templates
    image(key: string) {
        "images/{key}"
    }
}

Unlike KV fields, no specific type is necessary for an R2 field declaration, as the actual value is never queried and buffered into memory in the application layer.

Read more about R2 bindings in the Environment chapter.

Defining an R2 Field

Note

R2 is used to store large unstructured data. For this reason, Cloesce will not query and buffer the full value of an R2 field into the worker runtime. Instead, only a HEAD request is made to R2 to check for existence and retrieve metadata.

A field in a Model may reference an R2 bindings template to define an R2 field:

r2 MyBucket {
    // ...templates
    image(key: string) {
        "images/{key}"
    }
}

model Image {
    route {
        id: string
    }

    r2 MyBucket::image(id) {
        my_image
    }
}

The above snippet defines a Model Image with an R2 field my_image that is stored in the bucket MyBucket under the key “images/{id}”, where {id} is a placeholder that will be replaced with the actual value of the id route field when accessing R2.

See information about route fields in the KV fields chapter.

Generated Types

Backend

Since Cloesce does not fetch the actual value of an R2 field into the application layer, the Cloudflare standard R2ObjectBody type is used for all R2 fields in the generated backend code.

Each R2 Bucket will also generate a corresponding namespace with helper functions template, get, put, and list, with similar signatures to the KV helper functions generated for KV namespaces.

Frontend

It is possible to serialize the r2object type (or a Model field under the r2 block).

Cloesce will send a subset of the full R2 HEAD response metadata back to the frontend, including the key, version, size, etag, httpEtag, uploaded timestamp, and any custom metadata defined on the R2 object. This allows you to work with R2 objects in the frontend without having to fetch the full object data.

export interface R2Object {
  key: string;
  version: string;
  size: number;
  etag: string;
  httpEtag: string;
  uploaded: Date;
  customMetadata?: Record<string, string>;
}

Navigation Fields

Note

Navigation Fields only work in matching backings. For example, if you have a Durable Object backed Model and a D1 backed Model, you cannot create navigation fields that span across them.

Any Model may navigate to a Worker backed Model, but Worker backed Models cannot navigate to other backings (D1 or Durable Objects).

Defining foreign key relationships between your Models sets SQL constraints to maintain data integrity, but it doesn’t give you an easy way to access related data.

Navigation fields allow to set 1:1 and 1:M relationships, hydrated by the Cloesce ORM

One-to-One Relationship

Non SQLite Models

Given a relationship where a Person has one Profile, we can add a navigation field to the Person Model that allows us to access the related Profile directly:

model Profile {
    route {
        personId: string
    }

    // ... data
}

model Person {
    route {
        id: string
    }

    nav Profile::personId(id) {
        profile
    }
}

Here, the nav Profile::personId(id) block inside the Person Model tells Cloesce to create a navigation field called profile on the Person Model.

When you query for a Person, Cloesce will automatically populate the profile property with the corresponding Profile instance based on the matching value of Person’s id and Profile’s personId.

Tip

If Profile were to have no route fields, this syntax would be valid:

model Person {
  nav Profile { profile }
}

SQLite Models

Note

Foreign key relationships and navigation fields are defined independently, but for SQLite backed models, navigation fields rely on the presence of a foreign key relationship to function.

Cloesce uses the foreign key relationship to determine how to populate the navigation field with the correct related data.

Given the same relationship where a Person has one Profile, but this time both are SQLite backed, we can define the navigation field like so:

model Profile for MyDb {
    primary {
        id: int
    }
}

model Person for MyDb {
    primary {
        id: int
    }

    foreign Profile::id {
        profileId
    }

    nav Profile::id(profileId) {
        profile
    }
}

In this example, Person has a foreign key relationship to Profile through the profileId field. The nav Profile::id(profileId) block inside the Person Model tells Cloesce to create a navigation field called profile on the Person Model.

When you query for a Person, Cloesce will automatically populate the profile property with the corresponding Profile instance based on the matching value of Person’s profileId and Profile’s id.

One-to-Many Relationship

Note

Only a SQLite backed Model can have a 1:M relationship in the schema.

Let’s say we want Person to have any number of Dogs. We can achieve this with a one-to-many relationship:

model Dog for Db {
    primary {
        id: int
    }

    foreign Person::id {
        ownerId
    }
}

model Person for Db {
    primary {
        id: int
    }

    nav Dog::ownerId {
        dogs // 1:M nav field!
    }
}

In this example, Dog has a foreign key relationship to Person through the ownerId field. On the Person Model, we declare a navigation field dogs that references the Dog::ownerId foreign key. Cloesce will populate the dogs property with an array of all Dog instances that have an ownerId matching the Person’s id.

Data Sources

Before diving into APIs, it’s important to understand how Models are actually retrieved and manipulated in Cloesce.

This chapter provides a reference for how to write Data Sources in Cloesce, which are the building blocks for all data retrieval in your application.

Data Sources Overview

If you query an instantiated API endpoint of a Model, you may notice that Cloesce will leave undefined values or empty arrays in deeply nested compositions with other Models.

This is intentional, and an effect of Data Sources.

What are Data Sources?

Data Sources are Cloesce’s response to the overfetching and recursive relationship challenges when modeling relational databases with object-oriented paradigms.

Every Data Source is composed of an Include Tree, along with get, list, and save operations. Include Trees are necessary to determine which fields to hydrate when fetching data for a given Model.

For example, in the Model definition below, how should Cloesce know how deep to go when fetching a Person and their associated Dog?

model Dog for Db {
    primary {
        id: int
    }

    foreign Person::id {
        ownerId
    }

    nav Person::id(ownerId) {
        owner
    }
}

model Person for Db {
    primary {
        id: int
    }

    nav Dog::ownerId {
        dogs
    }
}

// => { id: 1, dogs: [ { id: 1, owner: { id: 1, dogs: [ ... ] } } ] } ad infinitum

If we were to follow this structure naively, fetching a Person would lead to fetching their Dog, which would lead to fetching the same Person again, and so on, resulting in an infinite loop of data retrieval.

Default Data Source

Every Model will come with a Default Data Source (called Default) that Cloesce will use whenever an operation does not explicitly specify a Data Source to use.

Default Include Tree

To prevent overfetching (and infinite loops), the Default Data Source will only join:

• Any R2 and KV fields
• All 1:1 Navigation Fields
• The near side of all 1:M Navigation Fields

Include Trees

To determine which fields to hydrate, Cloesce uses a construct called the Include Tree. An Include Tree is a recursive structure that represents the relationships between Models and their fields, and is used by Cloesce to determine how to fetch data for a given Model.

For example, in the Person and Dog Models above, the Default Data Source’s Include Tree would join only the field dogs on Person, but on the Dog Model, it would join owner, and then finally dogs.

// Include Tree for Person
include {
    dogs
}

// Include Tree for Dog
include {
    owner {
        dogs
    }
}

Custom Data Sources

It is likely that the Default Data Source generated by Cloesce will not fit all of your needs for data retrieval. For example, you may want to fetch a Person without their associated Dog, or fetch a Profile without its avatar R2 field.

Cloesce allows you to define custom Data Sources for any Model, which can include any combination of KV, R2, and Navigation Fields, in addition to custom SQL queries for D1 backed Models.

Defining a Data Source

Note

By default, any scalar property (i.e. SQLite columns) will be included by all Data Sources. They cannot be excluded.

Data Sources can be defined with a source block. In the inner include block, you can specify all fields to include in that Data Source, including R2, KV, and Navigation Fields.

source WithDogsOwnersDogs for Person {
    include {
        dogs {
            owner {
                dogs {
                    // ... could keep going!
                }
            }
        }
    }
}

Overriding the Default Data Source

The Default Data Source for a Model can be overridden by giving a source block the name Default, changing the default behavior of that Model when it is hydrated without a specified Data Source:

// Override the default to be empty
source Default for Person {
    include {}

    // ...methods
}

Get Method

Each time Cloesce needs to hydrate an instance of a D1 backed Model, it requires a Data Source with a get method defined.

If you do not define a get method, Cloesce will use a default get-by-id implementation. Otherwise, you can define a custom get method on any Data Source:

source ByName for Person {
    include {}

    get([instance] name: string)
}

A stub will be generated for the get method above, which you can then fill in with custom logic for fetching a Person by their name instead of their id.

instance tag

When Cloesce returns a Model to the client, the client receives a fully instantiated instance of that Model with methods that invoke the backend API.

Because the client has all necessary data to identify that instance (e.g. the primary key), it is not always necessary to manually attach identifying parameters to each API method.

To tell Cloesce that a parameter is already available on the client instance (i.e. it is a field of the Model), you can use the instance tag in your get method parameters. This allows you to omit that parameter when calling the API method on the client, and Cloesce will automatically pull it from the instance data and pass it to the backend.

source ByName for Person {
    include {}

    get([instance] name: string)
}

If instance were to be omitted in the example above, the generated API method would require a name parameter to be passed in, even though the client instance already has that data available.

List Method

While get is used during hydration, the list method is used only in the generated CRUD list endpoint for that Model. If you do not define a list method, Cloesce will generate one for you which performs seek pagination based on the primary key:

source Default for Person {
    include {}

    list(last_id: int, limit: int)
    // ... implemented with (for SQLite models):
    // SELECT * FROM Person WHERE id > $last_id ORDER BY id ASC LIMIT $limit
}

include Expansion

The include block specifies all fields to be included when that Data Source is used, and is required for Cloesce to know how to hydrate the instance. If omitted, Cloesce will use the Default Include Tree for that Model.

When hydrating an instance of a Model, Cloesce converts the include block (called an Include Tree) into a SQL query with the necessary joins to retrieve all specified fields in as few queries as possible.

The transpiled SQL query can be used within a Data Source’s get or list method, allowing you to write custom data retrieval logic while still leveraging the power of Cloesce’s include trees to handle complex relationships between Models.

For example:

source WithDogsOwnersDogs for Person {
    include {
        dogs {
            owner {
                dogs {}
            }
        }
    }

    [inject Db]
    list(owner_id: int, limit: int)
}

// main.ts
const WithDogsOwnersDogs = clo.Person.WithDogsOwnersDogs.impl({
  async list(env, owner_id, limit) {
    const stmt = env.Db.prepare(
      `WITH included AS (${this.selectQuery})
                SELECT * FROM included
                    WHERE "dogs.owner.id" = ?1
                    LIMIT ?2`,
    ).bind(owner_id, limit);

    return clo.Person.Orm.list(env, { query: stmt, include: this.tree });
  },
});

See more information on how ${this.selectQuery} is transpiled in the ORM Reference.

Internal Data Source

A Data Source may have no use on the client, and only be used internally by Cloesce or invoked by the developer using the Cloesce ORM. In this case, the Data Source can be marked as internal, which will prevent Cloesce from generating CRUD methods for that Data Source.

[internal]
source InternalOnly for Person {
    // ...
}

APIs

Defining the structure and source of your data with Models and Data Sources is only the first step in building a full stack application with Cloesce. To create a complete application, you also need to define how an outside client (frontend, third party service, etc.) can interact with your application and its data.

This chapter covers how to:

• Define REST APIs
• Utilize generated CRUD APIs
• Inject dependencies into APIs
• Validate API inputs with Runtime Validation
• Define Services and Plain Old Objects

REST APIs

Cloesce Models are not just about defining data and how to hydrate it; they also allow you to define how that data can be accessed and manipulated through APIs.

By defining an API for a Model, you can specify REST endpoints that are generated as backend stubs and client methods, routed by the Cloesce runtime.

Defining an API

Given some Model, we can define an API for it like so:

model Person for Db {
    primary {
        id: int
    }
}

api Person {
    get by_id(id: int) -> Person
    post create(name: string) -> Person
    delete del(id: int)
    put update(id: int, name: string) -> Person
    patch update_name(id: int, name: string) -> Person
}

The above code defines an API for the Person Model:

All of the above methods are static, meaning they are called in the namespace of a Model, but do not need to hydrate an instance of that Model.

Transpiled Code

After running cloesce compile, the above API definition could be implemented in TypeScript as follows:

import * as clo from "@cloesce/backend.js";

export const Person = clo.Person.impl({
  by_id(id) {
    // ...
  },

  create(name) {
    // ...
  },

  del(id) {
    // ...
  },
});

While the backend code must be implemented manually, the frontend client methods are generated automatically by Cloesce based on the API definition:

// .cloesce/client.ts
export class Person {
  id: number;

  static async by_id(id: number): Promise<HttpResult<Person>> {
    // ...
  }

  static async create(name: string): Promise<HttpResult<Person>> {
    // ...
  }

  static async del(id: number): Promise<HttpResult<void>> {
    // ...
  }
}

Instance Methods

With Cloesce, you can skip the step of manually hydrating and validating a Model instance. By passing the self keyword in the API method parameters, Cloesce will automatically hydrate the instance from the relevant data source and pass it as an argument to the method. For example:

model Person for Db {
    primary {
        id: int
    }
}

api Person {
    get myself(self) -> Person
}

The above code defines an API method GET /Person/:id/myself, which can be implemented in TypeScript:

import * as clo from "@cloesce/backend.js";

export const Person = clo.Person.impl({
  myself(self) {
    // `self` is an instance of `clo.Person.Self` that has been automatically hydrated by Cloesce
    return self;
  },
});

Using a Custom Data Source

By default, all API methods will use the Default Data Source to hydrate the self instance. However, you can specify a custom data source with the source tag:

model Person for Db {
    primary {
        id: int
    }

    r2 Bucket::avatars(id) {
        avatar
    }
}

source WithoutAvatar for Person {
    include {
        // Empty!
    }
}

api Person {
    get myself([source WithoutAvatar] self) -> Person
}

In the above code, the myself API method will use the WithoutAvatar data source to hydrate the self instance, which excludes the avatar field. This allows you to have different API methods that return different subsets of the Model’s data based on the data source used.

Execution Context

Durable Objects do not define just an area for storing data, but a single threaded execution context. Any method may be executed in the context of a Durable Object using Dependency Injection. For example:

durable CounterDo {
    shard {
        tenant: string
    }
}

model Counter {}
api Counter {
  [inject CounterDo(tenant)]
  put increment(tenant: string) -> int
}

Because increment injects an instantiated instance of CounterDo, any code within increment will be executed in the context of that Durable Object, allowing you to safely manipulate data stored in that Durable Object without worrying about race conditions.

Important

Only one instance of a Durable Object can be injected into a method at a time, since each instance represents a single threaded execution context.

Note

If a Durable Object has no shard keys, it is effectively a singleton, and can be injected as:

[inject Global()]
put method()

Note

Injecting the Durable Object namespace is different than injecting an instance of that durable object. For example, [inject CounterDo] would inject the namespace, allowing you to create and manage instances of that Durable Object within your method, but not execute code within the context of any particular instance.

Data Source Execution Context

Any method of a Data Source can also be executed in the context of a Durable Object by using the inject tag within that Data Source. The default implementation of a Data Source for some Durable Object backed Model will always be executed in the context of that Durable Object.

Additionally, passing self to an API method will utilize the context defined in the get method of the Data Source. This means that if the get method of the Data Source is executed in the context of a Durable Object, then any API method that hydrates self from that Data Source will also be executed in that context. For example:

source Default for Counter {
    [inject CounterDo(tenant)]
    get([instance] tenant: string)
}

source OutsideContext for Counter {
    get([instance] tenant: string)
}

api Counter {
    // Executed inside of CounterDo
    get myself(self) -> Counter

    // Executed outside of CounterDo
    get outside([source OutsideContext] self) -> Counter
}

Important

An instantiated method inherits its execution context from the get method of its Data Source, so it must not also inject one explicitly. Doing so is a compile error:

api Counter {
    // Error: `self` already runs inside CounterDo via the Data Source's `get`
    [inject CounterDo(tenant)]
    get myself(self, tenant: string) -> Counter
}

Streams

Cloesce buffers the full body of an incoming request by default, which is suitable for most use cases. However, for certain scenarios such as file uploads or real-time data processing, you may want to handle the request body as a stream.

To define a streaming API method, you can use the stream type in the API definition:

model File {
    primary {
        id: int
    }
}

api File {
    post upload(file: stream) -> File
    get download(id: int) -> stream
}

The above code defines two API methods for the File Model:

• POST /File/upload - Accepts a streaming file upload and returns a File instance
• GET /File/download - Returns a streaming response for downloading a file by its ID

The implementation of the upload method would need to handle the incoming stream appropriately by inspecting the ReadableStream passed in as the file parameter. Similarly, the download method would need to return a stream that can be consumed by the client for downloading the file.

HttpResult

Both the backend and frontend utilize the HttpResult type to represent the result of a REST API call. This type encapsulates the success or failure of the API call, along with any relevant data or error information.

The HttpResult type is defined as follows:

export class HttpResult<T = unknown> {
  public constructor(
    public ok: boolean,
    public status: number,
    public headers: Headers,
    public data?: T,
    public message?: string,
    public mediaType?: MediaType,
  ) {}

  /**
   * Return some OK result with the given status, data, and headers.
   */
  static ok<T>(status: number, data?: T, init?: HeadersInit): HttpResult<T>;

  /**
   * Return a failure result with the given status, message, and headers.
   * No body may be attached.
   */
  static fail(status: number, message?: string, init?: HeadersInit): HttpResult<never>;
}

For example, with the following schema:

model Garfield for Db {
    primary {
        id: int
    }
}

api Garfield {
    get by_id(id: int) -> Garfield
}

The implementation of the by_id method could return an HttpResult like so:

import * as clo from "@cloesce/backend.js";

export const Garfield = clo.Garfield.impl({
  by_id(id) {
    const today = new Date();
    const isMonday = today.getDay() === 1;

    if (isMonday) {
      return HttpResult.fail(503, "Garfield hates Mondays");
    }

    return HttpResult.ok(200, { id });
  },
});

CRUD Generation

Note

The delete operation is not currently supported, but will be added in a future release.

Creating the same CRUD operations for each Model can be tedious. Cloesce provides a way to automatically generate these operations based on your Model definitions and Data Source configurations.

For every public Data Source defined on a Model, Cloesce will utilize the get, save, and list methods of that Data Source to generate CRUD API endpoints for that Model.

Get

By default, the get operation retrieves a single record by its primary key, shard fields and route fields. For example:

[crud get]
model Person for PersonDo(tenant) {
    primary {
        id: int
    }
}

source Custom for Person {
    get(special_id: int)
}

The above schema will generate two API methods:

• GET /Person/$get: Accepts arguments tenant and id, hydrates with the Default Data Source, and returns a Person instance if a record is found

• GET /Person/$get_Custom: Accepts argument special_id, hydrates with the Custom Data Source, and returns a Person instance if a record is found

List

Important

The list operation can only be used if your Model does not have any route fields.

The list operation retrieves multiple records. By default, it will use a seek based pagination strategy. For example:

[crud list]
model Person for Db {
    primary {
        id: int
    }
}

source OffsetPagination for Person {
    list(offset: int, limit: int)
}

The above schema will generate two API methods:

• GET /Person/$list: Accepts arguments limit and lastSeen_id, hydrates with the Default Data Source, and returns a paginated list of Person instances

• GET /Person/$list_OffsetPagination: Accepts arguments offset and limit, hydrates with the Custom Data Source, and returns a paginated list of Person instances

Save

The save operation creates or updates any record within a Data Source’s include tree.

The only parameter save accepts is a partial Model instance, which is an object that may contain a subset of the Model’s fields. For example:

[crud save]
model Person for Db {
    primary {
        id: int
    }
}

The client could then invoke a method like:

export class Person {
  // ...
  static async $save(model: DeepPartial<Person>): Promise<HttpResult<Person>> {
    // ...
  }
}

const result = await Person.$save({
  id: 1,
  name: "Alice",
});

R2 Fields

If your Model contains an R2 field, the save operation will not be able to accept any data for that field, since the ORM is designed only for JSON serializable data. To work around this, you can define a custom instance method on your Model that accepts a stream parameter:

model Person {
    route {
        id: int
    }

    r2 Bucket::photos(id) {
        avatar
    }
}

api Person {
    [inject Bucket]
    post upload_photo(self, photo: stream)
}

import * as clo from "@cloesce/backend.js";

export const Person = clo.Person.impl({
  async upload_photo(self, env, photo) {
    await env.Bucket.photos.put(photo);
  },
});

Dependency Injection

Any API method may optionally inject Environment Bindings, or define custom object bindings to inject.

This allows you to easily access resources such as D1 databases, KV namespaces, R2 buckets, and more within your API implementations without needing a globally scoped environment object.

Injecting Environment Bindings

To inject an Environment Binding, add the inject tag to the API method and specify the name of the binding you want to inject:

d1 { Db }

r2 Bucket { }

vars {
    SECRET: string
}

model Person {
    route{
        id: int
    }
}

api Person {
    [inject Db, Bucket, SECRET]
    get do_stuff(self) -> Person
}

In the above code, the backend stub generated for the do_stuff API method will pass in a parameter env of the type:

{
  Db: D1Database;
  Bucket: Bucket;
  SECRET: string;
}

Defining Custom Inject Bindings

In addition to Environment Bindings, you can also define your own custom Inject bindings. This is useful for cases where you want to inject a resource that is not part of the environment, or if you want to perform some custom logic before injecting the resource.

To define some custom structure to be injected, you can use the inject keyword in your API definition:

inject {
    YouTubeApi
    OpenAiClient
}

// ... and then inject as usual:
api Person {
    [inject YouTubeApi, OpenAiClient]
    get do_stuff(self) -> Person
}

Unlike Environment Bindings, custom Injections require an explicit implementation:

import * as clo from "@cloesce/backend.js";

class YouTubeApi extends clo.YouTubeApi {
  // Add custom methods or properties!
  constructor() {
    super();
  }
}

export default {
  async fetch(request: Request, env: clo.Env): Promise<Response> {
    const app = clo.cloesce();
    app.register(new YouTubeApi());

    return await app.run(request, env);
  },
};

Runtime Validation

When an HTTP request is made to the Cloesce Router, the incoming data will first be matched to an existing API implementation, and then validated against the respective API defined in the Cloesce Schema.

Each type is validated in accordance with the rules defined in the Type Reference. If any validation errors occur, a 400 Bad Request response will be returned with details about the validation errors.

In addition to this, several Validator Tags are also supported for more complex validation scenarios.

Overview

Validator Tags can be applied to any field (i.e. it follows the syntax field: type) in a Model, API parameter, or Data Source parameter.

A foreign key field will automatically inherit all validators from the field it references. For example:

model User {
    primary {
        [gt 0]
        id: int
    }
}

model Post {
    primary {
        id: int
    }

    foreign (User::id) {
        user_id
    }
}

In the above code, the user_id field in the Post Model will automatically have the [gt 0] validator applied to it, since it is a foreign key referencing the id field in the User Model.

Numerical Validators

These validators apply to the int and real types:

String Validators

These validators apply to the string type:

Cloesce uses Rust’s regex crate for regular expression validation and evaluation. A regex pattern is provided as a regex literal:

[regex /^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/]
email: string

Plain Old Objects

In addition to Models, Cloesce supports Plain Old Objects (POOs) for structured data that doesn’t require database backing, such as data transfer objects (DTOs) or view models.

POOs are defined with the poo keyword and can have fields just like Models, but they lack the ORM and API capabilities that Models have.

Defining a POO

To define a POO, you can use the following syntax:

poo PersonDTO {
    id: int
    name: string
    age: int
}

The above code defines a POO called PersonDTO with three fields: id, name, and age. You can use this POO in your API definitions, data sources, or anywhere else you need to represent structured data without the overhead of a full Model.

POO Composition

POOs can also be composed of other POOs, allowing you to create complex data structures. For example:

poo GraphNode {
    id: int
    value: string
    children: array<GraphNode>
}

In the above code, the GraphNode POO has a field children which is an array of GraphNodes, allowing you to represent tree-like structures.

Services

It is reasonable to have an API endpoint that doesn’t quite fit into a Model’s namespace, or that needs to interact with multiple Models and Data Sources.

To accomodate this use case, Cloesce allows you to define a Model that has no data associated with it, and only serves as a namespace for API methods. This is commonly referred to as a “Service”.

Defining a Service

A service is a model block with empty braces:

model FooService {}

api FooService {
    get do_foo() -> string
}

The implementation is the same as for any other model:

import * as clo from "@cloesce/backend.js";

const FooService = clo.FooService.impl({
  do_foo() {
    return "foo";
  },
});

ORM Reference

Warning

The ORM is subject to change as new features are added.

Cloesce is responsible for hydrating your Models with data from various sources, in the same manner a traditional ORM hydrates objects from data in a database.

Additionally, Cloesce aims to replace boilerplate CRUD operations with simple save, get, and list methods on your Models.

The internal tools that Cloesce uses to accomplish this are available for the developer to use as well, allowing you to write custom SQL queries while still leveraging the power of Cloesce’s schema.

Generated Backend ORM

Two namespaces are placed on every Model in the generated backend code:

1. GeneratedSource
2. Orm

GeneratedSource

The GeneratedSource namespace contains transpiled representations of each Data Source defined for that Model. For example, the namespace for the Model WeatherReport:

// ...types and impls omitted for brevity
    export namespace GeneratedSource {
        export const Default = {
            tree: { weatherEntries: {} },
            selectQuery: `SELECT ... FROM "WeatherReport" ...`,

            getQuery(env, id: number): D1PreparedStatement {...},
            async get(env, id: number) {...},

            listQuery(env, lastSeen_id: number, limit: number): D1PreparedStatement {...},
            async list(env, lastSeen_id: number, limit: number) {...},

            async save(env, model: DeepPartial<WeatherReport.Self>) {...},
        }
    }

Tip

Data Source implementations access these via this, e.g. this.selectQuery and this.tree. See Custom Data Sources for an example.

Orm

While the GeneratedSource namespace exposes methods scoped to a defined Data Source, the Orm namespace contains lower level methods for hydrating and mapping Model instances. Each method defaults its Include Tree to the Default Data Source’s tree.

These methods are especially useful when you need to write custom queries but still want to leverage the schema and hydration capabilities of Cloesce.

Note

For Durable Object backed Models, save, get, and hydrate take the injected Durable Object as their first argument (instead of env), since hydration must occur within that object’s execution context.

More information on how the lower level building blocks work can be found in the next section.

Using the Base ORM Methods

The Orm namespace generated on each Model wraps the Orm class exported from the cloesce package. That class exposes the primitives select, map, and hydrate directly, each backed by an inner WebAssembly implementation. These take the Model’s Meta (exported on every generated Model) as their first argument.

select

Orm.select generates a SQL SELECT statement with LEFT JOINs and column aliases to retrieve all properties of a Model according to an Include Tree.

For example, given the Models Boss, Person, Dog, and Cat, where Boss has many Persons, and Person has one Dog and one Cat:

// ...
model Boss for Db {
    primary {
        id: int
    }

    nav Person::bossId {
        persons
    }
}

source WithAll for Boss {
    include {
        persons {
            dogs
            cats
        }
    }
}

Orm.select(Boss.Meta, null, Boss.GeneratedSource.WithAll.tree) produces:

SELECT
    "Boss"."id" AS "id",
    "Person_1"."id" AS "persons.id",
    "Person_1"."bossId" AS "persons.bossId",
    "Dog_2"."id" AS "persons.dogs.id",
    "Dog_2"."personId" AS "persons.dogs.personId",
    "Cat_3"."id" AS "persons.cats.id",
    "Cat_3"."personId" AS "persons.cats.personId"
FROM "Boss"
LEFT JOIN "Person" AS "Person_1"
    ON "Boss"."id" = "Person_1"."bossId"
LEFT JOIN "Dog" AS "Dog_2"
    ON "Person_1"."id" = "Dog_2"."personId"
LEFT JOIN "Cat" AS "Cat_3"
    ON "Person_1"."id" = "Cat_3"."personId"

Note the specific column aliases in the SELECT statement. Not only are these valuable to the map method, but they can be used in tandem with a common table expression to simplify the process of writing custom SQL queries with complex relationships:

WITH included AS (
    -- Generated SQL from Orm.select goes here
)
SELECT * FROM included WHERE "persons.dogs.name" = 'Fido'

map

Orm.map takes the results of a D1 SELECT query and attempts to reconstruct the object graph based on the column aliases. This is useful when you need to write custom SQL but still want to leverage the ORM’s hydration capabilities.

const result = await env.Db.prepare(
  `
    WITH included AS (
        -- Generated SQL from Orm.select goes here
    )
    SELECT * FROM included WHERE "persons.dogs.name" = 'Fido'
`,
).all();

const mapped = Orm.map(Boss.Meta, result, Boss.GeneratedSource.WithAll.tree);

While these mapped objects are now full JavaScript objects with the correct relationships, they are not yet hydrated according to the Model definition. For example, if the Model has any KV or R2 fields, those properties will not yet be populated.

hydrate

Orm.hydrate takes some base object (e.g. an element from map) and fetches any KV or R2 properties to return a fully populated Model instance. Additionally, it instantiates objects like Dates and Blobs according to the Model definition.

Since the generated Boss.Orm.hydrate wrapper already binds the Model’s Meta and accepts env directly, prefer it over the raw cloesce package call:

const mapped = Orm.map(Boss.Meta, result, Boss.GeneratedSource.WithAll.tree);
const hydrated = await Boss.Orm.hydrate(env, mapped[0], Boss.GeneratedSource.WithAll.tree);