"The Next Era of Software Architecture Is Data-First"

Software architecture has always been organized around behavior. You design services, write functions, define APIs. Data is a consequence — something that gets passed around, stored and analyzed. Observability is instrumented on top. Experimentation is wired in sideways. Analytics are a separate system entirely.

This isn't a tooling problem. It's a paradigm problem.

We've been building systems where code is primary and data is a side effect. The next era inverts that relationship.

What Data-First Actually Means

In a data-first architecture, the data model is the architecture. You don't design services and then figure out what data they produce. You define the records that matter — what they contain, how they flow, who produces them, who consumes them and the system's behavior emerges from that definition.

This isn't a new idea in theory. It's just never been tractable in practice, because no single primitive has existed that could represent data flow coherently from a microcontroller's flash memory to a Kubernetes pod in the cloud.

AimDB changes that. One codebase, three buffer primitives, the same typed API from MCU to cloud. Open source.

Why AimDB Is the Right Foundation

AimDB's core insight is simple: all data movement fits one of three patterns.

• SPMC Ring — for high-frequency streams where multiple consumers need independent reads. Sensor telemetry. Interaction events. Logs. This is what Kafka, Kinesis and every event streaming system is built on, expressed as a single typed buffer.
• SingleLatest — for state where only the current value matters. Feature flags. Configuration. UI state. This is what Redis, feature flag systems and config stores hold, in one primitive.
• Mailbox — for commands where the latest instruction wins. Device control. RPC. Actuation. This is what task dispatchers and control loops carry — only the latest instruction matters.

These aren't IoT-specific patterns. They're universal. Every data-driven system, from a microcontroller to a distributed Kubernetes service, is built from these three primitives. AimDB just makes them explicit, typed and portable across the entire hardware stack.

The Gap Between Today and Tomorrow

Most teams today treat data-driven development as a process layer on top of their architecture: add analytics, run experiments, look at dashboards, adjust behavior. The data is collected after the system runs.

In the next era, the data model is defined before the system is built and the system is derived from it.

The Architecture Graph

Every AimDB deployment has an implicit record graph: nodes are typed records, edges are producer and consumer relationships, buffer types are edge semantics. This graph doesn't describe your system's architecture — it is the running artifact.

Making this graph explicit is the next step — introspectable at runtime, queryable by tools and visible to anyone working with the system. Consider what it means in practice:

Observability becomes automatic. A record that exists is observable by definition. You don't instrument your system for metrics, the buffer itself has metrics. Every producer and consumer relationship is declared, not discovered.

Synchronization becomes declarative. You don't build a sync layer between your MCU, edge gateway and cloud backend. You declare a record with MQTT connector metadata on its key and the same typed data flows across all three environments.

Cross-cutting concerns derive from the schema. Instead of adding observability libraries, feature flag SDKs and experiment frameworks as separate integrations, they become properties of records — declared once, applied everywhere.

What It Looks Like In Code

// Data contracts — plain structs, portable across MCU/edge/cloud
#[derive(Clone, Debug, Serialize, Deserialize)]
struct CheckoutEvent {
    step: String,
    variant: String,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
struct LogCheckout {
    step: String,
    variant: String,
    timestamp: u64,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
struct ExperimentVariant {
    name: String,
    group: String,
    received_at: u64,
}

// Record keys — each variant maps to a buffer slot and an MQTT topic
#[derive(RecordKey, Clone, Copy, PartialEq, Eq, Debug)]
#[key_prefix = "events.checkout."]
enum CheckoutKey {
    #[key = "step"]
    #[link_address = "mqtt://events/checkout/step"]
    Step,
    #[key = "log"]
    Log,
}

#[derive(RecordKey, Clone, Copy, PartialEq, Eq, Debug)]
enum ExperimentKey {
    #[key = "checkout_flow"]
    #[link_address = "mqtt://config/experiment/checkout"]
    CheckoutFlow,
}

// Wiring — buffer semantics, MQTT direction, and data flow in one place
builder.configure::<CheckoutEvent>(CheckoutKey::Step, |reg| {
    reg.buffer(BufferCfg::SpmcRing { capacity: 1000 })
        .source(|ctx, producer| async move { /* emit checkout events */ })
        .link_to("mqtt://events/checkout/step")           // outbound: publish events
        .with_serializer(|ctx, e| Ok(serde_json::to_vec(e)?))
        .finish();
});

// Transform — enrich CheckoutEvent with a timestamp
builder.configure::<LogCheckout>(CheckoutKey::Log, |reg| {
    reg.buffer(BufferCfg::SpmcRing { capacity: 1000 })
        .transform::<CheckoutEvent>(CheckoutKey::Step, |ctx, event| {
            LogCheckout {
                step: event.step,
                variant: event.variant,
                timestamp: ctx.time().now(),              // enrich with receive time
            }
        })
        .finish();
});

builder.configure::<ExperimentVariant>(ExperimentKey::CheckoutFlow, |reg| {
    reg.buffer(BufferCfg::SingleLatest)                   // only the current variant matters
        .link_from("mqtt://config/experiment/checkout")   // inbound: receive assignments
        .with_deserializer(|ctx, data| {                  // context-aware: access runtime
            let mut variant: ExperimentVariant = serde_json::from_slice(data)?;
            variant.received_at = ctx.time().now();        // stamp with local receive time
            Ok(variant)
        })
        .finish();
});

The Rust type system enforces correctness at compile time. The buffer semantics enforce flow guarantees at runtime. The connector metadata wires everything to your existing infrastructure without an integration layer.

Ready to try data-first? Full examples are in the repo. Star it, clone it, run it. · Live Demo

Who This Is For

If you're building edge computing systems in Rust that span hardware boundaries — MCUs, edge gateways, cloud services — and you're tired of maintaining three different data representations, three different sync mechanisms and three different observability setups for what is conceptually one system, AimDB is built for you.

If you're building product software and you want experimentation, observability and behavioral analytics to be structural properties of your architecture rather than instrumentation bolted on afterward, this paradigm is built for you.

If you believe that the way we currently think about software — code first, data second — is going to look as dated in twenty years as procedural programming looks now, we're building toward the same future.

Get Involved

AimDB is open source, Apache 2.0 and early. The core is stable. The API is real. If any of the above resonates — come build with us.

Star AimDB on GitHub · Join the discussion · Read the docs

---

Graphics and spell checks in this post were created with the help of an LLM.