Making Calls¶

Suspending calls¶

JevClient implements the JevApi interface. Its calls are suspend functions, so call them from a coroutine:

// JevApi calls are suspend functions: call them from a coroutine.
val result = jev.ask(Triage, state = "Our dashboard has been down for an hour.")

query and ask are extension functions on JevApi. Import them with import com.pambrose.jev4k.query and import com.pambrose.jev4k.ask; IDEs add these automatically.

// evaluate is the one network call everything else builds on.
val result = jev.evaluate(JsonPrimitive("Refund me now!"), Triage.questions, model = null)

Blocking calls¶

jev.blocking mirrors every call without coroutines, for scripts, main, tests and Java callers:

// Outside coroutines (scripts, main, Java callers), use the blocking mirror.
JevClient().use { jev ->
    val result = jev.blocking.ask(Triage, state = "Our dashboard has been down for an hour.")
    println(result[Triage.team].choice)
    println(jev.blocking.models().map { it.name })
}

Blocking calls block the calling thread. Don't use them from inside a coroutine.

Choosing a model¶

Every call takes an optional model, which overrides the client's defaultModel for that request:

// Override the model per call. Pin a versioned model once you've tuned thresholds against it,
// because an alias such as jev-latest can move to a newer release.
val pinned = jev.ask(Triage, state = "The API returns 500s.", model = "jev-1.13.0")
println("answered by ${pinned.model}")

// List the model names this account can use.
jev.models().forEach { println("${it.name} (${it.releaseDate}): ${it.description}") }

jev-latest always points at the newest stable model, so its answers can shift when TypeSafe ships a release. Once you've tuned thresholds against a model, pin its version (for example jev-1.13.0) and move to a new one deliberately.

Concurrency¶

Suspending calls make it easy to process many states at once. Bound the parallelism, though: TypeSafe's examples found that about 8 concurrent requests on a shared key already hit rate limits.

// Run many requests concurrently, but bound the fan-out: shared keys hit rate limits at around 8 at once.
suspend fun triageBatch(
    jev: JevApi,
    tickets: List<String>,
    parallelism: Int = 4,
): List<Team> {
    val permits = Semaphore(parallelism)
    return coroutineScope {
        tickets
            .map { ticket -> async { permits.withPermit { jev.ask(Triage, state = ticket)[Triage.team].choice } } }
            .awaitAll()
    }
}

Rate-limited requests are retried automatically (see Retries & Errors), but staying under the limit is faster than being retried.

Before reaching for concurrency, check whether the questions could go in one request instead. Questions about the same state are answered in parallel for the cost of one request; see Speculative Fan-Out.

Designing for testability¶

Depend on the JevApi interface rather than JevClient, and pass it in:

// Depend on the JevApi interface, not JevClient, and pass it in.
class TicketRouter(
    private val jev: JevApi,
) {
    suspend fun queueFor(ticket: String): String =
        when (jev.ask(Triage, state = ticket)[Triage.team].choice) {
            Team.BILLING -> "billing"
            Team.TECHNICAL -> "engineering"
            Team.SALES -> "sales"
        }
}

// In production, wire in the real client; elsewhere, any JevApi implementation will do.
fun productionRouter(): TicketRouter = TicketRouter(JevClient())

query and ask both go through JevApi.evaluate, so a fake or mock that implements evaluate covers all three, without a network. models() is the interface's other network call; stub it too if your code lists models. To exercise the real client without a network, give it a Ktor MockEngine through the engine setting.

Build what the fake returns with jevResult(body, questions), which maps a response body exactly as the client does, and jevApiException(status) for the error path. Recording a real response and replaying it keeps a fixture honest, and a malformed one still raises JevResponseValidationException.