Custom Data Sources

It is likely that the Default Data Source generated by Cloesce will not fit all of your needs for data retrieval. For example, you may want to fetch a Person without their associated Dog, or fetch a Profile without its avatar R2 field.

Cloesce allows you to define custom Data Sources for any Model, which can include any combination of KV, R2, and Navigation Fields, in addition to custom SQL queries for D1 backed Models.

Defining a Data Source

Note

By default, any scalar property (i.e., any D1 standard column) will be included by all Data Sources. They cannot be excluded.

Data Sources can be defined with a source block. In the inner include block, you can specify all fields to include in that Data Source, including R2, KV, and Navigation Fields.

source WithDogsOwnersDogs for Person {
    include {
        dogs {
            owner {
                dogs {
                    // ... could keep going!
                }
            }
        }
    }
}

Overriding the Default Data Source

The Default Data Source for a Model can be overridden by giving a source block the name Default, changing the default behavior of that Model when it is hydrated without a specified Data Source:

// Override the default to be empty
source Default for Person {
    include {}
}

Get Method

Each time Cloesce needs to hydrate an instance of a D1 backed Model, it requires a Data Source with a get method defined.

If you do not define a get method, Cloesce will use a default get-by-id implementation. Otherwise, you can define a custom get method on any Data Source:

source ByName for Person {
    include {}

    sql get([instance] name: string) {
        "
        SELECT * FROM Person WHERE name = $name
        "
    }
}

`instance` tag

When Cloesce returns a Model to the client, the client receives a fully instantiated instance of that Model with methods that invoke the backend API.

Because the client has all necessary data to identify that instance (e.g. the primary key), it is not always necessary to manually attach identifying parameters to each API method.

To tell Cloesce that a parameter is already available on the client instance (i.e. it is a field of the Model), you can use the instance tag in your get method parameters. This allows you to omit that parameter when calling the API method on the client, and Cloesce will automatically pull it from the instance data and pass it to the backend.

source ByName for Person {
    include {}

    sql get([instance] name: string) {
        "
        SELECT * FROM Person WHERE name = $name
        "
    }
}

If instance were to be omitted in the example above, the generated API method would require a name parameter to be passed in, even though the client instance already has that data available.

List Method

While get is used during hydration, the list method is used only in the generated CRUD list endpoint for that Model. If you do not define a list method, Cloesce will generate one for you which performs seek pagination based on the primary key:

source Default for Person {
    include {}

    sql list(last_id: int, limit: int) {
        "
        SELECT * FROM Person WHERE id > $last_id ORDER BY id ASC LIMIT $limit
        "
    }
}

Custom list methods can be defined as well:

source ByName for Person {
    include {}

    sql list(last_name: string, limit: int) {
        "
        SELECT * FROM Person WHERE name > $last_name ORDER BY name ASC LIMIT $limit
        "
    }
}

`include` Expansion

Each Data Source must define an include block, even if that block is empty. The include block specifies all fields to be included when that Data Source is used, and is required for Cloesce to know how to hydrate the instance.

When hydrating an instance of a Model, Cloesce converts the include block (called an Include Tree) into a SQL query with the necessary joins to retrieve all specified fields in as few queries as possible.

The transpiled SQL query can be used within a Data Source’s get or list method, allowing you to write custom data retrieval logic while still leveraging the power of Cloesce’s include trees to handle complex relationships between Models.

For example:

source WithDogsOwnersDogs for Person {
    include {
        dogs {
            owner {
                dogs {}
            }
        }
    }

    sql get([instance] id: int) {
        "
        WITH included AS ($include)
        SELECT * from included WHERE id = $id
        "
    }

    sql list(owner_id: int, limit: int) {
        "
        WITH included AS ($include)
        SELECT * from included WHERE [dogs.owner_id] = $owner_id LIMIT $limit
        "
    }
}

See more information on how $include is transpiled in the ORM Reference.

Internal Data Source

A Data Source may have no use on the client, and only be used internally by Cloesce or invoked by the developer using the Cloesce ORM. In this case, the Data Source can be marked as internal, which will prevent Cloesce from generating a client API method for that Data Source.

[internal]
source InternalOnly for Person {
    // ...
}

Keyboard shortcuts

Cloesce