Sampling
MCP 2026-07-28 removed sampling/createMessage as a capability-driven
server→client request and re-homed the ability onto
MRTR
as an input-request kind — already deprecated, matching the spec's own
12-month lifecycle. The APIs on this page carry #[deprecated] and exist for
migration; call sites need #[allow(deprecated)].
For new code, prefer a host-provided tool over borrowing the client's model.
The Model Context Protocol (MCP) provides a standardized way for servers to request LLM sampling (“completions” or “generations”) from language models via clients. This flow allows clients to maintain control over model access, selection, and permissions while enabling servers to leverage AI capabilities—with no server API keys necessary. Servers can request text, audio, or image-based interactions and optionally include context from MCP servers in their prompts.
Important mental model
- Server requests sampling
- Client decides:
- which model to use
- whether sampling is allowed
- how tools are executed
- Server never owns API keys and never talks to LLMs directly
neva::types re-exports the sampling types only on client / legacy-spec
builds, so on a plain server-full build they are not in the prelude.
Import them from their own module — the snippets below do, as does the
upstream examples/sampling:
use neva::types::sampling::{CreateMessageRequestParams, SamplingMessage};
Basic Usage
To use sampling, inject Context into your tool handler and call the sample()
method with a prompt and a stable replay key.
use neva::prelude::*;
use neva::types::sampling::CreateMessageRequestParams;
#[tool]
async fn generate_weather_report(mut ctx: Context, city: String) -> Result<String, Error> {
let params = CreateMessageRequestParams::new()
.with_message(format!("What's the weather in {city}?"))
.with_sys_prompt("You are a helpful assistant.");
// Round 1 unwinds the handler with `input_required` and a
// `sampling/createMessage` envelope; round 2 replays the completion.
#[allow(deprecated)]
let result = ctx.sample("weather", params).await?;
Ok(format!("{:?}", result.content))
}
Because sampling rides the MRTR substrate, the handler re-runs from the
top on every round. Guard anything expensive or externally visible above
the sampling point with ctx.memo, ctx.once, or ctx.on_commit — the same
primitives that cover elicitation.
A server may only ask for a kind the client declared: registering a sampling
handler is what makes a client set clientCapabilities.sampling. Asking an
undeclared client reports an error rather than stalling the round-trip.
If you already have an appropriate prompt template declared in your MCP server, you may use the prompt() method of Context instead of passing formatted string.
Configure Create Message Request
The СreateMessageRequestParams struct provides methods to configure:
- Temperature
- System prompt
- Token limits
- Model preferences
use neva::prelude::*;
use neva::types::sampling::{CreateMessageRequestParams, ModelPreferences};
let model_pref = ModelPreferences::new()
.with_hints(["claude-4.5-sonnet", "gpt-5"])
.with_cost_priority(0.3)
.with_speed_priority(0.8)
.with_intel_priority(0.5);
let params = CreateMessageRequestParams::new()
.with_message(format!("What's the weather in {city}?"))
.with_sys_prompt("You are a helpful assistant.")
.with_max_tokens(1000)
.with_temp(0.2)
.with_pref(model_pref);
Model preferences are hints, not guarantees.
The client may:
- Ignore them
- Map them to a different model
- Apply additional policies
Tool Use
If the client supports the sampling.tools capability, server can provide a list of tools for LLM to use during sampling. You may do it via with_tools() method:
Tools are always executed by the server, never by the client or the model.
use neva::prelude::*;
use neva::types::sampling::CreateMessageRequestParams;
let Some(tool) = ctx.find_tool("get_weather").await else {
return Err(ErrorCode::MethodNotFound.into());
};
let params = CreateMessageRequestParams::new()
.with_message(format!("What's the weather in {city}?"))
.with_sys_prompt("You are a helpful assistant.")
.with_tools([tool]);
Context additionally has the tools(), find_tool() and find_tools() methods that could be helpful to fetching tools metadata for client.
Configure the tool choice
By default the with_tools() set the toolChoice for LLM to auto. However you may change it with the with_tool_choice() method.
use neva::prelude::*;
use neva::types::sampling::{CreateMessageRequestParams, ToolChoiceMode};
let Some(tool) = ctx.find_tool("get_weather").await else {
return Err(ErrorCode::MethodNotFound.into());
};
let params = CreateMessageRequestParams::new()
.with_message(format!("What's the weather in {city}?"))
.with_sys_prompt("You are a helpful assistant.")
.with_tools([tool])
.with_tool_choice(ToolChoiceMode::Required);
The ToolChoiceMode struct can be:
Auto- Model decides whether to call tools (default).Required- Model must call at least one tool.None- Model must not call any tools.
Handle Sampling Loop
Below is a reference implementation of a sampling loop with tool execution. Most real-world MCP servers follow this pattern.
The sample() method returns CreateMessageResult. You should inspect its stop_reason and continue sampling until a terminal reason is reached.
use neva::prelude::*;
use neva::types::sampling::{CreateMessageRequestParams, SamplingMessage, StopReason, ToolChoiceMode};
#[tool]
async fn generate_weather_report(mut ctx: Context, city: String) -> Result<String, Error> {
let Some(tool) = ctx.find_tool("get_weather").await else {
return Err(ErrorCode::MethodNotFound.into());
};
let mut params = CreateMessageRequestParams::new()
.with_message(format!("What's the weather in {city}?"))
.with_sys_prompt("You are a helpful assistant.")
.with_tools([tool]);
let mut step = 0;
loop {
// Each iteration needs its **own** replay key — a shared key would
// replay the first completion forever instead of advancing the loop.
#[allow(deprecated)]
let result = ctx.sample(format!("sample-{step}"), params.clone()).await?;
if result.stop_reason == Some(StopReason::ToolUse) {
// Getting the tool use requests from sampling response
let tools: Vec<ToolUse> = result.tools()
.cloned()
.collect();
// Logging them for the "context" as assistant messages
let assistant_msg = tools
.iter()
.fold(SamplingMessage::assistant(), |msg, tool| msg.with(tool.clone()));
// Calling tools that LLM requested — behind `memo`, so they run
// once. Without it every earlier step's tools would execute again
// on each later round, because the handler restarts from the top.
let tool_results = ctx
.memo(format!("tools-{step}"), async { Ok(ctx.use_tools(tools).await) })
.await?;
// Logging the tools results as user messages
let user_msg = tool_results
.into_iter()
.fold(SamplingMessage::user(), |msg, result| msg.with(result));
// Creating the params for the next step with the previous context and tool results
params = params
.with_message(assistant_msg)
.with_message(user_msg)
.with_tool_choice(ToolChoiceMode::None);
step += 1;
} else {
// Stopping if we get a reason different from tool use
return Ok(format!("{:?}", result.content));
};
}
}
Each sampling step must include:
- Assistant messages with tool calls
- User messages with tool results
This mirrors LLM training data and allows the client to reconstruct full context.
In production code you should always:
- Limit the number of sampling iterations
- Handle unexpected stop reasons
Each iteration is a full MRTR round-trip, so it also costs a client
re-issue. Clients cap those with McpOptions::with_max_mrtr_rounds.
And every side effect in the loop body needs a per-iteration key, not
just the sample call: the handler restarts from the top on each round, so
an unguarded ctx.use_tools would re-execute every earlier step's tools
each time round.
When Not to Use Sampling
Avoid sampling when:
- The task is deterministic
- No natural language reasoning is required
- A regular tool or function call is sufficient
Learn By Example
A complete working example is available here.