Rust Quickstart

Get a working HTAP engine up in under 30 lines. Requires tokio::main.

Add Dependencies

[dependencies]
yoda  = "1"
arrow = "58"
tokio = { version = "1", features = ["full"] }

End-to-End Example

The snippet below is a complete, runnable program. It creates an in-memory engine, registers a users table, writes three rows through OLTP, manually syncs CDC events to the OLAP mirror, and then runs an aggregate query that is automatically routed to the OLAP engine.

use std::sync::Arc;
use arrow::datatypes::{DataType, Field, Schema};
use yoda::{HtapConfig, HtapEngine, TableSchema};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Configure the engine.
    //    olap_in_memory = true  →  no files written (ideal for tests/demos)
    //    sync_interval = None   →  manual sync via sync_now()
    let config = HtapConfig {
        oltp_path: "quickstart.db".to_string(),
        olap_in_memory: true,
        ..HtapConfig::default()
    };

    // 2. Start the engine (creates the SQLite file + OLAP context).
    let mut engine = HtapEngine::new(config).await?;

    // 3. Declare the table schema.
    //    Arrow schema is the canonical type information for both OLTP and OLAP.
    let schema = TableSchema {
        name: "users".to_string(),
        arrow_schema: Arc::new(Schema::new(vec![
            Field::new("id",    DataType::Int64, false),
            Field::new("name",  DataType::Utf8,  true),
            Field::new("age",   DataType::Int32, true),
        ])),
        primary_key: vec!["id".to_string()],
    };
    engine.register_table(schema).await?;

    // 4. Write rows — SqlParserRouter sees INSERT, sends it to SQLite OLTP.
    engine.execute("INSERT INTO users VALUES (1, 'Alice', 30)", &[]).await?;
    engine.execute("INSERT INTO users VALUES (2, 'Bob',   25)", &[]).await?;
    engine.execute("INSERT INTO users VALUES (3, 'Carol', 35)", &[]).await?;

    // 5. Flush CDC events from _yoda_cdc_log to the OLAP mirror.
    //    (With sync_interval set, this happens automatically in the background.)
    let result = engine.sync_now().await?;
    println!("Synced {} events", result.events_processed);

    // 6. Run an analytical query — GROUP BY triggers OLAP routing.
    let batches = engine.query("SELECT COUNT(*) AS n FROM users").await?;
    println!("{} batch(es) returned, first row count = {:?}", batches.len(), batches[0].column(0));

    // 7. Graceful shutdown — cancels the background loop if one was started.
    engine.shutdown().await;
    Ok(())
}

Explicit Routing with query_with_hint

By default query uses SqlParserRouter to decide where to run each statement. To override the router, call query_with_hint:

use yoda::{HtapEngine, QueryHint};

// Force a SELECT to the OLAP engine (e.g. to benchmark a DuckDB aggregate)
let batches = engine
    .query_with_hint("SELECT name FROM users WHERE id = 1", QueryHint::ForceOlap)
    .await?;

// Force a statement to OLTP (useful during schema migrations)
let batches = engine
    .query_with_hint("SELECT * FROM users LIMIT 10", QueryHint::ForceOltp)
    .await?;

Auto-routing rules

query routes aggregates, GROUP BY, HAVING, JOINs, window functions, CTEs, and subqueries to OLAP. Simple SELECT … WHERE pk = ? and all writes go to OLTP. The fallback (when AST parsing fails) is always OLTP so writes are never accidentally lost.

Automatic Background Sync

Instead of calling sync_now() manually, set sync_interval in the config and Yoda starts a background loop:

use std::time::Duration;
use yoda::HtapConfig;

let config = HtapConfig {
    oltp_path: "app.db".to_string(),
    olap_in_memory: false,
    olap_path: Some("app-olap".to_string()),
    sync_interval: Some(Duration::from_millis(200)),
    ..HtapConfig::default()
};

The loop runs until engine.shutdown().await is called, or until the process exits.

Next Steps

• Architecture — how the OLTP/OLAP split and CDC pipeline work
• Configuration Reference — every HtapConfig field explained
• Sync Modes — destructive mirror vs. SCD Type 2 temporal history
• OLAP Backends — DataFusion vs. DuckDB trade-offs