Skip to main content
skill CX Codex

mlkit-agent-api — Claude Code skill: what it does, how to install

The mlkit-agent-api Claude Code skill guides Android agent workflows built with the ML Kit GenAI Agent API. It applies to multi-turn chat, on-device tool calls, and GenAiException handling.

Apache-2.0 374 installs ★ 10,691 updated 2026-10-23

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add android/skills --skill mlkit-agent-api -a claude-code
Open on GitHub

What mlkit-agent-api does

The skill covers setting up an Agent and AgentRunner for on-device execution with ML Kit GenAI Agent API, Gemini Nano, and AICore. Its examples address multi-turn chat with AgentSession in a ViewModel, Android AppFunctions, and one-turn runs through runSingleTurnAsync.

It also describes session history, cleanup, and GenAiException handling. AppFunctions use the Gradle dependencies com.google.mlkit:genai-agent:1.0.0-alpha1 and androidx.appfunctions:appfunctions:1.0.0-alpha13. The guidance targets AICore Developer Preview and excludes cloud Gemini models and Vertex AI.

What it helps with

  • configure Agent and AgentRunner for on-device execution
  • keep multi-turn chat history in a ViewModel
  • connect Android AppFunctions as agent tools
  • handle GenAiException and close the session

Example prompts for the agent

  • Set up an on-device Android agent with the ML Kit GenAI Agent API.
  • Add multi-turn chat with AgentSession in a ViewModel and handle GenAiException.
  • Show how to connect AppFunctions and run a single turn with runSingleTurnAsync.

How to install mlkit-agent-api

1Skills CLI (Vercel)

npx skills add android/skills --skill mlkit-agent-api -a claude-code

2Manually: put the files in place

~/.claude/skills/mlkit-agent-api/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/mlkit-agent-api/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about mlkit-agent-api

How do I install mlkit-agent-api in Claude Code?

Run npx skills add android/skills --skill mlkit-agent-api -a claude-code. The skill file is at ~/.claude/skills/mlkit-agent-api/SKILL.md.

What does mlkit-agent-api do?

It guides Android implementations that use ML Kit GenAI Agent API for on-device agent workflows, including multi-turn chat, AppFunctions tool calls, and error handling.

What does mlkit-agent-api require?

It targets AICore Developer Preview and uses the Gradle dependencies com.google.mlkit:genai-agent:1.0.0-alpha1 and androidx.appfunctions:appfunctions:1.0.0-alpha13. Its scope is Gemini Nano rather than cloud Gemini models or Vertex AI.

Source text

SKILL.md · Apache-2.0
---
name: mlkit-agent-api
description: Builds autonomous on-device agentic workflows in Android apps using the
  ML Kit GenAI Agent API powered by Gemini Nano and AICore. Use this skill when implementing
  conversational agents, multi-turn chat with AgentSession in a ViewModel, autonomous
  tool calling using Android AppFunctions (@AppFunctionDeclaration and AppFunctionAgentTools.discover),
  ephemeral single-turn execution with runner.runSingleTurnAsync, or handling GenAiException
  error resilience. It solves the problem of orchestrating local reasoning loops,
  typed tool dispatch, and session persistence on-device with zero cloud latency and
  privacy preservation. Don't use for cloud-based Gemini models or Vertex AI, single-turn
  text generation without agent loops (suggest Prompt API), or non-Android platforms.
license: Complete terms in LICENSE.txt
metadata:
  author: Google LLC
  last-updated: '2026-11-10'
  keywords:
  - Android
  - ML Kit
  - GenAI
  - Agent API
  - Gemini Nano
  - AICore
  - AppFunctions
  - Tool Calling
  - Function Calling
  - Session Persistence
  - AgentSession
---

Targeting **AICore Developer Preview**. This skill provides authoritative
execution patterns and failure-prevention rules for on-device agentic workflows
in Android apps.

*** ** * ** ***

## 1. Prerequisites and setup

### Gradle dependencies

```kotlin
// app/build.gradle.kts
dependencies {
    // Core ML Kit GenAI Agent API
    implementation("com.google.mlkit:genai-agent:1.0.0-alpha1")

    // Android AppFunctions dependency for on-device tool declarations
    implementation("androidx.appfunctions:appfunctions:1.0.0-alpha13")
}
```
> \[!IMPORTANT\] Always use official SDK interfaces from
> `com.google.mlkit.genai.agent.*`, `androidx.appfunctions.*`,
> `com.google.mlkit.genai.prompt.*`, and `com.google.mlkit.genai.common.*`.
> Never declare local placeholder interfaces or fallback mocks.

*** ** * ** ***

## 2. Minimal agent and runner initialization

Configure an `Agent` with `Agent.builder` and wrap execution in `AgentRunner`:

```kotlin
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner

val agent = Agent.builder()
    .setName("OnDeviceAssistant")
    .setInstructions("You are a helpful on-device assistant.")
    .build()

val runner = AgentRunner(agent)
```

- `Agent.builder` defaults to `maxTurns = 10` automatically to prevent infinite tool loops.
- `AgentRunner` defaults to greedy decoding (`temperature = 0.0f`) for deterministic tool selection and schema compliance.
- `AgentRunner(agent)` automatically resolves application context using `MlKitContext` for AppFunction execution in Android apps. An explicit `AgentRunner(context, agent)` overload is available for headless tests or custom context scopes.

*** ** * ** ***

## 3. Multi-turn chat and lifecycle (ViewModel)

`AgentSession` preserves conversational history in memory and is **not
`Parcelable`** . To preserve dialogue history across Android configuration
changes (such as screen rotation), retain `session` inside an AndroidX
`ViewModel`.

When executing single-turn ephemeral queries, managing concurrent
multi-sessions, or requiring explicit RAM cleanup, you must follow the
guidelines in [Session lifecycle](references/session_lifecycle.md).

```kotlin
package com.example.app

import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentSession
import com.google.mlkit.genai.agent.textTokens
import com.google.mlkit.genai.common.GenAiException
import com.google.mlkit.genai.prompt.Content
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch

class ChatViewModel : ViewModel() {
    private val agent = Agent.builder()
        .setInstructions("You are a helpful on-device assistant.")
        .build()
    private val runner = AgentRunner(agent)
    private val session: AgentSession =
        runner.getOrCreateSession("chat_session")

    private val _uiState = MutableStateFlow<ChatUiState>(ChatUiState.Idle)
    val uiState: StateFlow<ChatUiState> = _uiState.asStateFlow()

    fun sendMessage(prompt: String) {
        viewModelScope.launch {
            _uiState.value = ChatUiState.Loading("")
            try {
                // Content.builder() required; runAsync takes Content
                val content = Content.builder().text(prompt).build()
                var fullResponse = ""
                session.runAsync(content)
                    // Extension function requiring textTokens import
                    .textTokens()
                    .collect { token ->
                        fullResponse += token
                        _uiState.value = ChatUiState.Loading(fullResponse)
                    }
                _uiState.value = ChatUiState.Success(fullResponse)
            } catch (e: GenAiException) {
                // Catch GenAiException to prevent app crashes on AI errors
                val err = e.localizedMessage ?: "AI error (${e.errorCode})"
                _uiState.value = ChatUiState.Error(err)
            }
        }
    }

    override fun onCleared() {
        super.onCleared()
        // Evict conversation session from RAM when ViewModel is destroyed
        session.close()
    }
}
```

### Essential lifecycle and coroutine rules

1. **`Content` object** : `session.runAsync` and `runner.runSingleTurnAsync` require a `Content` object. Always build it using `Content.builder().text(userPrompt).build()`.
2. **`textTokens` extension** : `textTokens` is a Kotlin extension function on `Flow<AgentEvent>`. Always include `import
   com.google.mlkit.genai.agent.textTokens`.
3. **Session cleanup** : Always invoke `session.close` in `ViewModel.onCleared` to evict conversational RAM and cancel pending tasks.
4. **`AgentEvent` stream** : When collecting `Flow<AgentEvent>` directly instead of `textTokens`, always include `import
   com.google.mlkit.genai.agent.AgentEvent`. Match against `AgentEvent.TextMessage(text, isPartial)`, `AgentEvent.FunctionCall(name, args)`, and `AgentEvent.FunctionResponse(name, response)`, and always include an `else
   -> {}` branch in `when (event)` expressions (`AgentEvent` is non-exhaustive for forward compatibility):

```kotlin
import com.google.mlkit.genai.agent.AgentEvent

session.runAsync(content).collect { event ->
    when (event) {
        is AgentEvent.FunctionCall -> {
            _uiState.update {
                it.copy(activeToolStatus = "Tool started: ${event.name}")
            }
        }
        is AgentEvent.FunctionResponse -> {
            _uiState.update {
                it.copy(activeToolStatus = "Tool completed: ${event.name}")
            }
        }
        is AgentEvent.TextMessage -> {
            _uiState.update {
                it.copy(responseText = it.responseText + event.text)
            }
        }
        else -> {} // Required: AgentEvent is non-exhaustive for forward compat
    }
}
```

*** ** * ** ***

## 4. Tool calling recipe

All tools used with `Agent` originate from Android AppFunctions
(`@AppFunctionDeclaration`), registered using the declarative path
(`enableAppFunctions(true)` or `addAppFunctionIds(...)`) or advanced discovery
(`AppFunctionAgentTools.discover(...)`).

When you need to restrict caller access levels for private in-app tools or wrap
discovered AppFunctions with human-in-the-loop (HITL) confirmation decorators,
you must consult [Tools and functions](references/tools_and_functions.md).

### Option A: Declarative AppFunctions path

Define tools declaratively in your app using AndroidX AppFunctions:

```kotlin
package com.example.app.tools

import androidx.appfunctions.AppFunctionDeclaration
import androidx.appfunctions.AppFunctionSerializable

@AppFunctionSerializable
data class CalendarEvents(val date: String, val events: List<String>)

class CalendarTools {
    /**
     * Fetches scheduled calendar appointments for a specific date in
     * YYYY-MM-DD format.
     *
     * @param date Date string in YYYY-MM-DD format
     */
    @AppFunctionDeclaration(isDescribedByKDoc = true)
    fun fetch_calendar(
        date: String
    ): CalendarEvents {
        return CalendarEvents(
            date = date,
            events = listOf(
                "Team Standup at 10:00 AM",
                "Product Review at 2:00 PM"
            )
        )
    }
}
```

Enable AppFunctions on `Agent.Builder` and initialize `AgentRunner`:

```kotlin
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner

val agent = Agent.builder()
    .setName("CalendarAssistant")
    .setInstructions("You assist with calendar queries.")
    // Automatically discovers AppFunctions in your app package
    .enableAppFunctions(true)
    // Optional: scope to specific function IDs or method names
    .addAppFunctionIds("fetch_calendar")
    .build()

val runner = AgentRunner(agent)
```

- **AppFunction Contract** : Tool methods take domain parameters directly (no framework context parameter required). Returns must be primitives or `@AppFunctionSerializable` data classes (`Map<String, Any?>` isn't supported for AppFunctions).
- **Restricting Callers with `ACCESS_LEVEL_SELF`** : AppFunctions don't have to be public to external callers. To restrict execution strictly to your app's internal agent (blocking external system assistants and IPC callers), annotate with `@OptIn(ExperimentalAppFunctionsApi::class)` and `@AppFunctionAccessLevel(level = AppFunctionMetadata.ACCESS_LEVEL_SELF,
  isCompatEnforcementEnabled = true)`.

### Option B: Advanced discovery and decorator pattern

For inspecting, filtering, or decorating discovered AppFunctions (such as
Human-in-the-Loop confirmation wrappers using `AgentTool(delegate)`) before
registering with the agent:

```kotlin
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentTool
import com.google.mlkit.genai.agent.AppFunctionAgentTools

// Discover tools (optionally scoped to function IDs)
val discoveredTools = AppFunctionAgentTools.discover(
    enabledAppFunctionIds = listOf("fetch_calendar")
)

// Optional: Wrap sensitive tools using AgentTool(delegate) decorator
class HitlConfirmationTool(
    delegate: AgentTool,
    private val onConfirm: suspend (
        toolName: String,
        args: Map<String, Any>
    ) -> Boolean
) : AgentTool(delegate) {
    override suspend fun execute(args: Map<String, Any>): Map<String, Any?> {
        if (!onConfirm(name, args)) {
            return mapOf("status" to "CANCELLED_BY_USER")
        }
        return checkNotNull(delegate).execute(args)
    }
}

val agent = Agent.builder()
    .setName("CalendarAssistant")
    .setInstructions("You assist with calendar queries.")
    .addTools(discoveredTools)
    .build()

val runner = AgentRunner(agent)
```
> \[!WARNING\] **Mutual exclusivity** : Configure AppFunctions using either the
> declarative path (`enableAppFunctions(true)` on `Agent.Builder`) or the
> advanced path (`AppFunctionAgentTools.discover` passed to `addTools`), but
> not both. Using both simultaneously triggers a warning and registers
> duplicate tool schemas.

### Deprecated annotations (@Tool and @Param)

The legacy `@Tool` and `@Param` annotations in
`com.google.mlkit.genai.schema.annotations.*`, along with
`AgentTool.fromAnnotatedObject`, `Agent.Builder.addTools(annotatedObject:
Any)`, and standalone `FunctionTool`, have been deprecated or removed. All tools
added to `Agent` must originate from
`@androidx.appfunctions.AppFunctionDeclaration` using
`Agent.Builder.enableAppFunctions(true)` or `AppFunctionAgentTools.discover`.

*** ** * ** ***

## 5. Ephemeral single-turn execution

For one-off tasks where conversation history isn't needed (such as summarizing
text, formatting data, or answering standalone questions), invoke
`runner.runSingleTurnAsync(content)` directly:

```kotlin
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.textTokens
import com.google.mlkit.genai.prompt.Content
import kotlinx.coroutines.flow.Flow

// Executes a single-turn query ephemerally without session persistence:
fun askQuickQuestion(runner: AgentRunner, question: String): Flow<String> {
    val content = Content.builder().text(question).build()
    return runner.runSingleTurnAsync(content).textTokens()
}
```

- `runner.runSingleTurnAsync(content)` automatically creates and evicts an ephemeral session upon Flow completion.
- **Cleanup reference** :
  - `session.close`: Releases session resources and cancels coroutines (call in `ViewModel.onCleared`).
  - `session.delete` or `runner.deleteSession(sessionId)`: Permanently deletes the session from the runner.

*** ** * ** ***

## 6. Error handling and GenAiException reference

Runtime failures during on-device execution (such as model unavailability or
exceeding maximum turn limits) are emitted or thrown as `GenAiException`:

```kotlin
import com.google.mlkit.genai.common.GenAiException
```

### Key error codes

- `GenAiException.ErrorCode.MAX_TURNS_EXCEEDED` (`-112`): The agent exceeded `maxTurns` (default 10) without completing the reasoning loop.
- `GenAiException.ErrorCode.NOT_AVAILABLE` (`8`): The Gemini Nano model or AICore service isn't available on this device.
- `GenAiException.ErrorCode.BUSY` (`9`): System or AICore is busy, or battery or thermal quota has been reached.

© Google LLC · Apache-2.0 · android/skills

Similar skills

All skills for Agent setup: memory, context, orchestration →
rc-revenuecat-api-quick-reference RevenueCat rc-revenuecat-api-quick-reference is a Claude Code skill and a reference for the RevenueCat Android SDK 10.x and REST API. The agent uses it to look up signatures and object fields while writing purchase or entitlement code. CX Codex +1 747installs testing-mobile-api-authentication mahipal The testing-mobile-api-authentication skill audits mobile API authentication and authorization, covering JWT, OAuth, token handling, sessions, IDOR, and privilege escalation. CX Codex +1 276installs sauce-api-mcp saucelabs sauce-api-mcp connects AI coding assistants to the Sauce Labs infrastructure. It lets agents inspect test logs, check cloud devices, and track tunnel status. CX Codex +1 ★ 10repo stars open-autoglm-phone-agent reason-machines open-autoglm-phone-agent — a skill for Claude Code, Codex, Cursor and other IDEs that teaches your agent to set up Open-AutoGLM, an AI phone agent framework controlling Android, HarmonyOS NEXT and iOS via the AutoGLM vision-language model. CX Codex +1 1,400installs firebase-ai-logic-basics Google LLC firebase-ai-logic-basics adds Firebase AI Logic to client apps across Android, iOS, Web, and Flutter to invoke Gemini models without spinning up a backend. CX Codex +1 146,900installs limrun-expo-development limrun-inc limrun-expo-development is a Claude Code skill that prepares and runs Expo and React Native apps on Limrun, with dev-client builds, a Metro tunnel and JS/TS iteration without repeated native rebuilds. CX Codex +1 45,300installs

From the same set36

The whole set →
litert-runtime litert-runtime walks through Kotlin code and Gradle setup for running .tflite models on Android with the CompiledModel API. Covers CPU and GPU inference, buffer reuse, preprocessing, and migration from the legacy TFLite stack. CX Codex +1 558installs wear-widgets wear-widgets — a skill that builds, migrates, and checks Wear OS widgets using Glance Wear and Remote Compose. It steps in when an Android project needs a new widget, a TileService migration, or a review of an existing implementation. CX Codex +1 544installs android-for-cars-parked-apps android-for-cars-parked-apps — a skill for Claude Code adapts existing Android video apps, games, and web browsers for parked use on Android Auto and Android Automotive OS. CX Codex +1 537installs litert-lm This Claude Code skill guides Android app work with on-device text language models. It covers chat, summarization, and extraction through LiteRT-LM. CX Codex +1 451installs projected-hardware The projected-hardware Claude Code skill explains how Android apps access hardware on Android XR glasses. It covers camera and microphone permissions, device connections, and audio routing. CX Codex +1 444installs glance-appwidget-updates-and-memory glance-appwidget-updates-and-memory — a Claude Code skill for updating Android AppWidgets built with Jetpack Glance and keeping their images within memory limits. It covers widget setup, state sync, and crash fixes. CX Codex +1 400installs ink-app-builder The ink-app-builder Claude Code skill adds stylus and touch drawing to Android apps with Jetpack Ink and Compose. It activates for drawing surfaces, stroke rendering, erasing, undo history, or stroke storage. CX Codex +1 387installs ink-custom-brush-builder ink-custom-brush-builder — a skill for Claude Code and Cursor builds custom brush definitions for Android apps using Jetpack Ink. Sets up tip shapes, multi-coat layers, and stylus behaviors. CX Codex +1 382installs

Project sponsors

Companies that keep this analytics open