Extensions to the Zig 0.16 standard library: fixed-capacity containers, strict parsers, IPv4 types, and the primitives std lacks.

The same helpers kept showing up copy-pasted across projects: strict number parsing, IPv4 and CIDR handling, fixed-capacity buffers, a bounded writer, a slab allocator. stdx is where they live. Each module is tested on its own, and the package has no dependencies.

Fixed-capacity containers: lists, queues, maps, sets, Bloom filters, and comptime lookup tables. The storage lives in the value, with no allocator and no deinit.
Strict parsing: fmt.atoi accepts one grammar for machine-written integers and rejects the rest.
Base32: a Crockford base32 codec for fixed-shape machine tokens. The decode itself is the validation, and every spelling outside the lowercase alphabet fails.
IP addresses: net.Ip4 and net.Prefix4, following Go's net/netip: classification, ordering, canonical prefixes, and overlap checks.
UUIDs: Uuid generates version 4 and version 7 values, formats them, and parses the canonical text form.
Timekeeping: clock reads with no Io value, a per-tick cached clock, wall time for log lines, and a log rate limiter.
Timers: a caller-driven set of deadlines over Io.Timestamp, with callbacks, removal by handle, and period re-arm without drift.
Logging transport: slot_ring.Mpsc moves log lines from worker threads to one consumer. Producers never block, and a refused claim drops the new message.
The module list covers the rest: Swiss-table maps, a slab allocator, rapidhash, secure random, and buffered stdio.

This guide explains how to choose modules and combine them. The API reference documents every symbol.

Getting started

Install

Add stdx to your project:

zig fetch --save git+https://github.com/baskingspot/stdx

Add the module to your build in build.zig:

const stdx = b.dependency("stdx", .{ .target = target, .optimize = optimize });
exe.root_module.addImport("stdx", stdx.module("stdx"));

stdx builds with Zig 0.16 only.

Import it

Import the root module once in each file. Then bind the modules that the file uses:

const stdx = @import("stdx");
const buffer = stdx.buffer;
const timex = stdx.time;

If the file uses only one module, import it directly:

const timex = @import("stdx").time;

Name the bindings

Name a binding after its module. If the file already has a std module with the same name, add an x suffix.

Binding	Why
`timex`	`std.time` is in scope in most files.
`fmtx`	`std.fmt` is in scope in most files.
`buffer`	std has no `buffer` module.
`stdio`	std has no `stdio` module.

The suffix keeps both modules readable at the call site. timex.now(.awake) is stdx. time.ns_per_ms is std.

This rule is for code that uses stdx. Inside stdx, a module binding never has a suffix.

Choose std or stdx

stdx does not replace std. Use std when std has what you need. Use stdx for the cases where std is too liberal, needs an Io value that the code does not have, or has no equivalent, such as a Bloom filter.

Strict parsing is an example. Use fmt.atoi for input with a machine grammar, such as an HTTP Content-Length header or a line from /proc. It rejects bases, underscores, and trailing characters. Use std.fmt.parseInt for input that people write, such as command-line flags and config values. There, 1_000_000 is easier to read.

Read the examples

Every example in this guide is a complete file in docs/examples/. The test step compiles and runs each one, so the code in this guide builds. Run them all:

just test

The API reference has the contract for each symbol. This guide has the recipes.

Modules

A module that extends a std namespace takes its name: mem, fmt, net, posix, heap, hash. A concept std lacks gets its own: bloom, swiss, set. A container family follows std.array_hash_map. The module names the concept, and the constructors inside are Custom, Auto, and String.

Module	Use it for
`base32`	Crockford base32 codec for fixed-shape machine tokens: lowercase-only `encode` and `decode`, where the decode is the validation.
`bloom`	Bloom filters with a fixed capacity.
`bounded_array_hash_map`	An insertion-ordered hash map with a fixed capacity and no allocator.
`BoundedWriter`	An `Io.Writer` over an allocated buffer with a hard byte limit. It fails at the limit instead of growing.
`buffer`	Append buffers with a fixed capacity. `Array` owns its storage. `Slice` borrows it.
`fmt`	Strict number parsing (`atoi`) and comptime hex decoding.
`hash`	`RapidHash`.
`heap`	`SlabAllocator`.
`mem`	Fixed-size byte values and memory scans.
`multi_array`	Struct-of-arrays types with the storage inside the value.
`net`	IPv4 addresses and CIDR prefixes, after Go's `net/netip`: parse, classify, order, step, and test containment. `Ip4Endpoint` holds the octets and port in 6 bytes.
`pages`	Release whole kernel pages back to the OS. Linux only.
`posix`	`getsockname`, `getpeername`, `setsockopt`, and `getsockopt` on raw socket handles.
`priority_queue`	A priority queue with a fixed capacity.
`random`	Secure random bytes without an `Io` value.
`set`	A bounded set with set algebra.
`slot_ring`	A bounded MPSC ring for log lines: producers never block, and one consumer drains. Linux only.
`static_map`	Lookup tables that the compiler builds from a literal.
`stdio`	Buffered stderr and stdout streams that never return an error.
`swiss`	Resizable Swiss-table hash maps.
`time`	Clock reads without an `Io` value, a cached clock, and a log rate limiter.
`timers`	A caller-driven set of timers over `Io.Timestamp`: arm and stop entries by handle, fire callbacks, and repeat on a period.
`Uuid`	UUID v4 and v7. Generate, parse, and format.

Choosing a container

Most stdx containers have a fixed capacity. The capacity is part of the type, the storage is inside the value, and no allocator is involved. One container, swiss, grows at runtime and uses an allocator.

Find the need in the table. Then read the section for that type in Fixed-capacity containers.

You need	Use	Notes
A list with a known maximum length	`buffer.Array`	Storage inside the value.
The same list over memory you already own	`buffer.Slice`	Same methods. The capacity is the slice length.
One array per struct field	`multi_array.Buffer`	Runtime length. `multi_array.Array` has a fixed length.
The smallest item first	`priority_queue.Custom`	A binary heap. You supply the comparison.
A map that keeps insertion order	`bounded_array_hash_map.Auto` or `.String`	Fixed capacity.
A set	`set.Auto` or `.String`	A map with no values, plus set algebra.
A constant lookup table	`static_map.Auto` or `.String`	Built at compile time from a literal. Not the same as `std.StaticStringMap`: see below.
A cheap "have I seen this" test	`bloom.Auto`	Can report a key that was never added.
A map that grows	`swiss.Auto` or `.String`	Uses an allocator. Order is not insertion order.

Pick a fixed capacity when

The maximum is a real limit that your program already enforces, such as a configured connection count.
The value must copy freely, such as a configuration struct.
The code must not allocate, such as a signal handler or an event loop that runs after startup.

Pick std or swiss when

The size depends on input that you do not control. A fixed capacity turns a large input into error.NoSpaceLeft.
The container is large and rarely full. A copy moves the whole capacity, used or not.
You need std.ArrayList features that the fixed types lack.

Check std first

std.StaticStringMap also builds a lookup table at compile time, but only for string keys. Build a table at compile time lists how static_map differs. Use static_map when the keys are not strings, or when you want a compile error for a duplicate key.
std.EnumMap covers dense enum keys.
std.PriorityQueue, std.ArrayList, and std.AutoArrayHashMap allocate. The stdx types of the same shape do not.

Fixed-capacity containers

A fixed-capacity container stores its elements in an array inside the value. The capacity is a comptime parameter, so it is part of the type. buffer.Array(u8, 16) and buffer.Array(u8, 32) are different types.

Three rules hold for every type on this page except swiss, which the last section covers:

No method takes an allocator, and no value needs a deinit.
An operation that does not fit returns error.NoSpaceLeft. It never grows.
A copy of the value is a full copy of the storage.

Build a list

buffer.Array is the base type. append and appendSlice return error.NoSpaceLeft when the item does not fit.

const std = @import("std");
const buffer = @import("stdx").buffer;

// The capacity is part of the type. The storage lives inside the value.
const Frame = buffer.Array(u8, 16);

// A length prefix and a payload, built with no allocator. A payload that
// does not fit is an error the caller handles.
fn encode(payload: []const u8) error{NoSpaceLeft}!Frame {
    var frame: Frame = .empty;
    try frame.append(@intCast(payload.len));
    try frame.appendSlice(payload);
    return frame;
}

// When the code has already checked the size, the AssumeCapacity pair
// skips the error path.
fn encodeChecked(payload: []const u8) ?Frame {
    if (payload.len + 1 > Frame.capacity) return null;
    var frame: Frame = .empty;
    frame.appendAssumeCapacity(@intCast(payload.len));
    frame.appendSliceAssumeCapacity(payload);
    return frame;
}

test "a payload that fits" {
    const frame = try encode("hello");
    try std.testing.expectEqualSlices(u8, "\x05hello", frame.constSlice());
}

test "a payload that does not fit" {
    try std.testing.expectError(error.NoSpaceLeft, encode("this payload is too long"));
    try std.testing.expectEqual(null, encodeChecked("this payload is too long"));
}

test "the checked path matches the fallible path" {
    const a = try encode("abc");
    const b = encodeChecked("abc").?;
    try std.testing.expectEqualSlices(u8, a.constSlice(), b.constSlice());
}

Each fallible method has an AssumeCapacity form. Use it only after the code has checked the size. In Debug and ReleaseSafe builds, a wrong assumption is a safety-checked panic.

Keep it in a struct

A fixed-capacity list is a good field in a configuration struct. The struct copies by value, and nothing needs to be freed.

const std = @import("std");
const stdx = @import("stdx");
const buffer = stdx.buffer;
const Ip4 = stdx.net.Ip4;

const Backend = struct {
    addr: Ip4,
    port: u16,
};

// A list field that copies with its struct. No allocator, no deinit, no
// ownership to track.
const BackendList = buffer.Array(Backend, 8);

const Config = struct {
    listen: Ip4 = .loopback,
    backends: BackendList = .empty,
};

test "a config copies by value" {
    var config: Config = .{};
    try config.backends.append(.{ .addr = .parseComptime("10.0.0.1"), .port = 5432 });

    // The copy owns its own list. A reload can edit one and keep the other.
    var next = config;
    try next.backends.append(.{ .addr = .parseComptime("10.0.0.2"), .port = 5432 });

    try std.testing.expectEqual(1, config.backends.constSlice().len);
    try std.testing.expectEqual(2, next.backends.constSlice().len);
}

test "the whole capacity travels with the value" {
    // A copy moves every byte of the storage, used or not. Keep the
    // capacity close to the real maximum.
    try std.testing.expect(@sizeOf(buffer.Array(u8, 4096)) >= 4096);
}

Choose the capacity close to the real maximum. A copy moves every byte of the storage, so a large capacity makes every copy, return, and pass by value expensive. A very large value can also overflow the stack. Put such a value in a global or in a struct that you allocate.

Use one constant for the limit. The command-line check and the type then agree about the maximum.

Borrow storage

buffer.Slice has the methods of buffer.Array, but it appends into memory that the caller owns. Use it when the memory already exists, such as a stack array or a region of a larger buffer.

const std = @import("std");
const buffer = @import("stdx").buffer;

// `Slice` has the API of `Array`, but over storage the caller owns. The
// capacity is a runtime value: the length of the slice.
test "append into caller storage" {
    var storage: [8]u8 = undefined;
    var out: buffer.Slice(u8) = .init(&storage);

    try out.appendSlice("abc");
    try std.testing.expectEqualStrings("abc", out.constSlice());

    // The bytes are in the caller's array, not in a copy.
    try std.testing.expectEqualStrings("abc", storage[0..3]);

    try std.testing.expectError(error.NoSpaceLeft, out.appendSlice("too long"));
}

The slice stores a pointer. Keep the storage alive for as long as the Slice is in use.

Order by priority

priority_queue.Custom is the binary heap of std.PriorityQueue with a fixed capacity. The comparison function returns .lt when the first argument must pop first.

const std = @import("std");
const priority_queue = @import("stdx").priority_queue;

const Timer = struct {
    deadline_ms: i64,
    id: u32,
};

// The comparator returns `.lt` when `a` must pop before `b`.
fn byDeadline(_: void, a: Timer, b: Timer) std.math.Order {
    return std.math.order(a.deadline_ms, b.deadline_ms);
}

// At most 64 pending timers. The queue never allocates.
const Timers = priority_queue.Custom(Timer, void, 64, byDeadline);

test "timers pop in deadline order" {
    var timers: Timers = .empty;
    try timers.push(.{ .deadline_ms = 300, .id = 1 });
    try timers.push(.{ .deadline_ms = 100, .id = 2 });
    try timers.push(.{ .deadline_ms = 200, .id = 3 });

    // Peek reads the next deadline without removing it.
    try std.testing.expectEqual(100, timers.peek().?.deadline_ms);

    try std.testing.expectEqual(2, timers.pop().?.id);
    try std.testing.expectEqual(3, timers.pop().?.id);
    try std.testing.expectEqual(1, timers.pop().?.id);
    try std.testing.expectEqual(null, timers.pop());
}

A queue made with .empty has an undefined context. If the comparison reads the context, make the queue with initContext.

Store key and value pairs

bounded_array_hash_map is the data structure of std.AutoArrayHashMapUnmanaged with a fixed capacity. Entries keep their insertion order. Use Auto for keys that std.hash.autoHash accepts and String for []const u8 keys.

const std = @import("std");
const bounded_array_hash_map = @import("stdx").bounded_array_hash_map;

// Keys and values live in arrays inside the value. Iteration follows
// insertion order.
const Names = bounded_array_hash_map.Auto(u32, []const u8, 2);

test "a full map refuses new keys" {
    var names: Names = .init;
    try names.put(10, "ten");
    try names.put(20, "twenty");

    // A new key does not fit.
    try std.testing.expectError(error.NoSpaceLeft, names.put(30, "thirty"));

    // An existing key still updates in place, and keeps its position.
    try names.put(10, "TEN");
    try std.testing.expectEqualStrings("TEN", names.get(10).?);
    try std.testing.expectEqualSlices(u32, &.{ 10, 20 }, names.keys());
}

When the map is full, put fails for a new key. It still updates an existing key.

Store a set

set is a map with no values and the usual set operations. The operations work in place and do not allocate. add returns true for a new key and false for a key that is already present. It returns error.NoSpaceLeft when a new key does not fit, and unite does the same when the union does not fit.

const std = @import("std");
const set = @import("stdx").set;

const Ports = set.Auto(u16, 8);

test "membership and set algebra" {
    var allowed: Ports = .init;
    for ([_]u16{ 80, 443, 8080 }) |port| _ = try allowed.add(port);

    var requested: Ports = .init;
    _ = try requested.add(443);
    _ = try requested.add(80);
    try std.testing.expect(requested.subsetOf(&allowed));

    _ = try requested.add(22);
    try std.testing.expect(!requested.subsetOf(&allowed));

    // Keep only what the allow list also holds.
    requested.intersect(&allowed);
    try std.testing.expectEqual(2, requested.count());
    try std.testing.expect(!requested.contains(22));
}

// `add` has three outcomes: newly added, already present, and no room.
test "a full set refuses a new key" {
    var ports: set.Auto(u16, 2) = .init;
    try std.testing.expect(try ports.add(80));
    try std.testing.expect(!try ports.add(80));
    try std.testing.expect(try ports.add(443));
    try std.testing.expectError(error.NoSpaceLeft, ports.add(8080));
}

test "unite fails when the union does not fit" {
    const Small = set.Auto(u16, 3);
    var a: Small = .init;
    var b: Small = .init;
    _ = try a.add(1);
    _ = try a.add(2);
    _ = try b.add(3);
    _ = try b.add(4);
    try std.testing.expectError(error.NoSpaceLeft, a.unite(&b));
}

Build a table at compile time

static_map builds a lookup table from a literal during compilation. The table is a constant. It has no startup cost, and a duplicate key is a compile error.

const std = @import("std");
const static_map = @import("stdx").static_map;

// The table is built at compile time from a literal. It has no runtime
// initialization, and a duplicate key is a compile error.
const Opcode = static_map.Auto(u8, []const u8, .{
    .{ 0x01, "push" },
    .{ 0x02, "pop" },
    .{ 0xff, "halt" },
});

test "lookup in a comptime table" {
    try std.testing.expectEqualStrings("halt", Opcode.get(0xff).?);
    try std.testing.expectEqual(null, Opcode.get(0x03));
    try std.testing.expect(Opcode.has(0x02));
    try std.testing.expectEqual(3, Opcode.count);
}

// The keys and values keep the order of the literal.
test "iteration follows the literal" {
    try std.testing.expectEqualSlices(u8, &.{ 0x01, 0x02, 0xff }, Opcode.keys);
}

std.StaticStringMap also builds a table at compile time, for string keys. static_map differs in these ways:

Key types: std takes []const u8 only. static_map.Auto takes key types that std.hash.autoHash accepts, such as u8, u32, and enums with an explicit u8 tag. A key with bits narrower than a byte, such as a u2, a bool, or an enum with three tags, is a compile error, because std hashes those bits differently at comptime than at runtime. Use static_map.Custom with your own hash for such keys. static_map.Custom takes your own hash and equality context, and static_map.String is the string-keyed form.
Lookup: std finds a key by its length, then compares it against each key of that length. static_map hashes the key and probes a table.
Duplicates: std accepts a duplicate key, and which of its values get returns is unspecified. static_map rejects a duplicate key at compile time.
Order: std orders keys() and values() by key length. static_map keeps the order of your literal.
What only std has: getLongestPrefix, which finds the longest key that starts a string, and init, which builds the map at runtime with an allocator. static_map has neither.

For string keys, use std unless you want one of those properties from static_map. For a dense enum key, std.EnumMap fits. For any other key type, std has no table of this kind.

Test membership cheaply

bloom is a Bloom filter. contains never misses a key that was added. It can report a key that was never added, at a known rate. The rate holds only while the filter holds the number of keys that you sized it for.

const std = @import("std");
const bloom = @import("stdx").bloom;

// Sized at compile time for 10,000 keys at about a 1% false-positive rate.
const Seen = bloom.Auto(u64, 10_000, 0.01);

test "a filter has no false negatives" {
    var seen: Seen = .empty;
    seen.add(42);
    try std.testing.expect(seen.contains(42));
}

test "the request is a target and actual_rate is the truth" {
    // The filter rounds its size up to a power of two, so the rate it
    // delivers can differ from the request.
    try std.testing.expect(Seen.actual_rate <= 0.01);
}

Store columns

multi_array keeps one array for each field of a struct. A loop over one field reads one contiguous array. Buffer has a runtime length. Array has a fixed length.

const std = @import("std");
const multi_array = @import("stdx").multi_array;

const Particle = struct {
    x: f32,
    y: f32,
    alive: bool,
};

// One array per field, with fixed capacity and a runtime length.
const Particles = multi_array.Buffer(Particle, 128);

test "a field is a contiguous slice" {
    var particles: Particles = .empty;
    try particles.append(.{ .x = 1, .y = 10, .alive = true });
    try particles.append(.{ .x = 2, .y = 20, .alive = false });

    // A loop over one field touches one array, not every struct.
    for (particles.items(.x)) |*x| x.* += 1;

    try std.testing.expectEqualSlices(f32, &.{ 2, 3 }, particles.constItems(.x));
    try std.testing.expectEqual(10, particles.get(0).y);
}

Grow when you must

swiss is the exception. It is a resizable hash map, so every call that can allocate takes the allocator, and the value does not store it. The map keeps the key slices that you give it, so the keys must outlive the map.

const std = @import("std");
const swiss = @import("stdx").swiss;

// The one container here that grows. It allocates, so every call that can
// allocate takes the allocator, and the value does not store it.
const Counts = swiss.String(u32);

test "a resizable map takes the allocator per call" {
    const gpa = std.testing.allocator;

    var counts: Counts = .init;
    defer counts.deinit(gpa);

    // 1,000 keys is far past any fixed capacity. The map grows.
    var key_buffer: [16]u8 = undefined;
    for (0..1000) |i| {
        const key = try std.fmt.bufPrint(&key_buffer, "key-{d}", .{i});
        // The map keeps the key slice, not a copy. A stack buffer would
        // dangle, so give the map keys that outlive it.
        const owned = try gpa.dupe(u8, key);
        errdefer gpa.free(owned);
        try counts.put(gpa, owned, @intCast(i));
    }
    defer {
        var it = counts.keyIterator();
        while (it.next()) |k| gpa.free(k.*);
    }

    try std.testing.expectEqual(1000, counts.count());
    try std.testing.expectEqual(42, counts.get("key-42").?);
}

Strict parsing

The std parsers accept more than one grammar. That is correct for input that people write. For input that a program writes, it hides errors. fmt.atoi accepts one grammar and rejects the rest.

Use it at a machine boundary, such as an HTTP header or a line from /proc. Use std.fmt.parseInt for values that people type, such as command-line flags.

Parse an integer

fmt.atoi accepts an optional sign (for signed types) and decimal digits. The whole string must be the number. It rejects base prefixes, underscores, spaces, and an empty string.

const std = @import("std");
const fmtx = @import("stdx").fmt;

// An HTTP Content-Length header is machine-written. It has one grammar:
// decimal digits and nothing else.
fn contentLength(value: []const u8) !u64 {
    return fmtx.atoi(u64, value);
}

test "plain digits parse" {
    try std.testing.expectEqual(1024, try contentLength("1024"));
}

test "everything else is an error" {
    // The whole string must be the number.
    try std.testing.expectError(error.InvalidCharacter, contentLength("1024 "));
    try std.testing.expectError(error.InvalidCharacter, contentLength("0x400"));
    try std.testing.expectError(error.InvalidCharacter, contentLength("1_000"));
    try std.testing.expectError(error.InvalidCharacter, contentLength(""));

    // A value too large for the type is a different error.
    try std.testing.expectError(error.Overflow, contentLength("18446744073709551616"));
}

test "std.fmt.parseInt reads input that people write" {
    // std accepts underscores. That suits a config value and is wrong for a
    // header.
    try std.testing.expectEqual(1000, try std.fmt.parseInt(u64, "1_000", 10));
}

An empty or malformed string returns error.InvalidCharacter. A number too large for the type returns error.Overflow.

Choose the grammar

fmt.atoiWith takes comptime options. Each set of options compiles to its own loop, so the choice has no runtime cost.

const std = @import("std");
const fmtx = @import("stdx").fmt;

// The options are comptime. Each configuration compiles to its own loop.
// This one is the JSON grammar for integers: no leading zeros.
fn jsonInt(text: []const u8) !u32 {
    return fmtx.atoiWith(u32, text, .{ .leading_zeros = .reject });
}

test "leading zeros" {
    try std.testing.expectEqual(0, try jsonInt("0"));
    try std.testing.expectEqual(42, try jsonInt("42"));
    try std.testing.expectError(error.InvalidCharacter, jsonInt("042"));

    // The default allows them, like std.fmt.parseInt.
    try std.testing.expectEqual(42, try fmtx.atoi(u32, "042"));
}

.overflow = .unchecked skips the overflow check. The caller promises that the value fits. A value that does not fit is a safety-checked panic in Debug and ReleaseSafe, and undefined behavior in ReleaseFast. fmt.atoiUnchecked is the shorthand.

Base32

base32 is a Crockford base32 codec for fixed-shape machine tokens. std has base64 and no base32. A token carries a binary key whose length is a positive multiple of 5 bytes. The token is exactly 8 * N / 5 lowercase symbols for N bytes, so it has no padding and a fixed length.

Both directions take the byte count as a comptime parameter. base32.encode(10, &key) returns a [16]u8, and base32.decode(10, &token) returns a [10]u8. base32.SymbolCount(N) names the token type for a field declaration. It rejects a count that is not a positive multiple of 5 at compile time.

The wire form is lowercase only. encode emits the lowercase Crockford alphabet, which excludes i, l, o, and u. decode rejects every byte outside that alphabet: uppercase letters fail, and the excluded letters fail in lowercase too. There is no case folding and no O/I/L alias table, so two spellings never decode to one key.

decode returns error.InvalidSymbol on the first bad byte. A token that decodes is valid by construction, so the decode itself is the validation. The caller runs no separate symbol check, and the fixed shape means the decode needs no length check either.

const std = @import("std");
const base32 = @import("stdx").base32;

// A token carries a binary key of a fixed shape. Sixteen lowercase
// Crockford symbols hold the ten bytes, with no padding.
test "a key round-trips through its token" {
    const key = [_]u8{ 0xDE, 0xAD, 0xBE, 0xEF, 0x00, 0x11, 0x22, 0x33, 0x44, 0x55 };
    const token = base32.encode(10, &key);
    try std.testing.expectEqual(@as(usize, 16), token.len);
    try std.testing.expectEqual(key, try base32.decode(10, &token));
}

// Decode is the validation: a symbol outside the lowercase alphabet is
// the only failure, and no separate check runs before it.
test "every other spelling is rejected" {
    // Uppercase is not a second spelling: no case folding, no aliases.
    try std.testing.expectError(error.InvalidSymbol, base32.decode(10, &"ABCDEFGHJKMNPQRS".*));
    // The excluded letters i, l, o, u are invalid in lowercase too.
    try std.testing.expectError(error.InvalidSymbol, base32.decode(10, &"uuuuuuuuuuuuuuuu".*));
}

IP addresses

net.Ip4 and net.Prefix4 follow Go's net/netip, with Zig names. They have the same semantics for classification, ordering, and prefix masking. They are plain values with no allocation.

Ip4 is four bytes with no port. std's Ip4Address carries a port, which a configuration field or an allow list does not need. Ip4 converts to and from the std types.

Parse an address

Ip4.parse accepts dotted-quad text and nothing else. Each error names the problem.

For a literal in code, use Ip4.parseComptime. It parses at compile time, and a typo is a compile error that quotes the string. Prefix4 has the same function. Ip4.init takes the four bytes directly, and Prefix4.init takes an address and a length.

const std = @import("std");
const Ip4 = @import("stdx").net.Ip4;

// An address has one canonical text form. Every other input is an error
// that names the problem.
test "parse accepts the canonical form" {
    const ip: Ip4 = try .parse("10.1.2.3");
    try std.testing.expectEqual(Ip4.init(.{ 10, 1, 2, 3 }), ip);
}

test "parse names what went wrong" {
    try std.testing.expectError(error.InvalidCharacter, Ip4.parse("example.com"));
    try std.testing.expectError(error.InvalidCharacter, Ip4.parse("::1"));
    try std.testing.expectError(error.Incomplete, Ip4.parse(""));
    try std.testing.expectError(error.NonCanonical, Ip4.parse("10.01.2.3"));
}

// A literal in code is parsed at compile time. A typo is a compile error
// that quotes the string, not a runtime failure.
test "parseComptime for literals" {
    const ip: Ip4 = .parseComptime("10.1.2.3");
    try std.testing.expectEqual(Ip4.init(.{ 10, 1, 2, 3 }), ip);
}

test "the well-known addresses are constants" {
    try std.testing.expectEqual(Ip4.parseComptime("127.0.0.1"), Ip4.loopback);
    try std.testing.expectEqual(Ip4.parseComptime("0.0.0.0"), Ip4.unspecified);
    try std.testing.expectEqual(Ip4.parseComptime("255.255.255.255"), Ip4.broadcast);
}

Classify an address

Ip4 has the netip predicates: isLoopback, isPrivate, isMulticast, isLinkLocalUnicast, isLinkLocalMulticast, isBroadcast, isUnspecified, and isGlobalUnicast.

isGlobalUnicast follows Go. A private or link-local address is global unicast, because it is routable. If the code means "public", it must exclude those classes.

const std = @import("std");
const Ip4 = @import("stdx").net.Ip4;

// Following Go's net/netip, a private or link-local address counts as
// global unicast: it is routable, just not on the public internet. Code
// that means "public" must say so.
fn isPublic(addr: Ip4) bool {
    return addr.isGlobalUnicast() and !addr.isPrivate() and !addr.isLinkLocalUnicast();
}

test "public means more than global unicast" {
    try std.testing.expect(isPublic(.parseComptime("8.8.8.8")));

    const private: Ip4 = .parseComptime("10.1.2.3");
    try std.testing.expect(private.isGlobalUnicast());
    try std.testing.expect(!isPublic(private));

    try std.testing.expect(!isPublic(.parseComptime("169.254.1.1")));
}

test "the other classes" {
    try std.testing.expect(Ip4.parseComptime("127.0.0.1").isLoopback());
    try std.testing.expect(Ip4.parseComptime("172.16.0.1").isPrivate());
    try std.testing.expect(!Ip4.parseComptime("172.32.0.1").isPrivate());
    try std.testing.expect(Ip4.broadcast.isBroadcast());
    try std.testing.expect(Ip4.unspecified.isUnspecified());
}

test "multicast and its link-local block" {
    const all_hosts: Ip4 = .parseComptime("224.0.0.1");
    try std.testing.expect(all_hosts.isMulticast());
    try std.testing.expect(all_hosts.isLinkLocalMulticast());
    try std.testing.expect(!all_hosts.isGlobalUnicast());

    // Multicast outside 224.0.0.0/24 is not link-local.
    const other: Ip4 = .parseComptime("239.1.2.3");
    try std.testing.expect(other.isMulticast());
    try std.testing.expect(!other.isLinkLocalMulticast());
}

Compare and step

order compares two addresses in numeric order. next and prev step by one address and return null at the ends of the space, so a loop cannot wrap.

const std = @import("std");
const Ip4 = @import("stdx").net.Ip4;

// Count the addresses from `first` to `last`, inclusive. `next` carries
// across octets and returns null past the last address, so the loop ends
// at 255.255.255.255 instead of wrapping.
fn countRange(first: Ip4, last: Ip4) u32 {
    var count: u32 = 0;
    var current = first;
    while (current.order(last) != .gt) {
        count += 1;
        current = current.next() orelse break;
    }
    return count;
}

test "a range that crosses an octet" {
    const first: Ip4 = .parseComptime("10.0.0.250");
    try std.testing.expectEqual(12, countRange(first, .parseComptime("10.0.1.5")));
}

test "a range that ends at the top of the space" {
    try std.testing.expectEqual(2, countRange(.parseComptime("255.255.255.254"), .broadcast));
}

test "order compares numerically" {
    const two: Ip4 = .parseComptime("2.0.0.0");
    try std.testing.expectEqual(.lt, two.order(.parseComptime("10.0.0.0")));

    const ten: Ip4 = .parseComptime("10.0.0.2");
    try std.testing.expectEqual(Ip4.parseComptime("10.0.0.1"), ten.prev());
}

Work with prefixes

Prefix4 is a network address and a length in bits. parse is strict. It rejects a missing slash, a length above 32, and a length with leading zeros.

contains tests an address against the prefix.
masked zeroes the host bits and returns the canonical form.
overlaps tests whether two prefixes cover a common address.
isSingleIp tests for a /32, a prefix of exactly one address.
Ip4.prefix builds the prefix of a given length around an address.
A prefix formats with {f}, like an address.

const std = @import("std");
const net = @import("stdx").net;
const Ip4 = net.Ip4;
const Prefix4 = net.Prefix4;

// A prefix from config text can carry host bits. `masked` returns the
// canonical form, so two spellings of one network compare equal.
test "canonical form" {
    const written: Prefix4 = .parseComptime("10.1.2.3/20");
    try std.testing.expectEqual(Prefix4.parseComptime("10.1.0.0/20"), written.masked());
}

test "containment works on any prefix length" {
    const subnet: Prefix4 = .parseComptime("10.1.16.0/20");
    try std.testing.expect(subnet.contains(.parseComptime("10.1.31.255")));
    try std.testing.expect(!subnet.contains(.parseComptime("10.1.32.0")));
}

// Two rules that cover a common address are usually a config mistake.
test "overlap detection" {
    const a: Prefix4 = .parseComptime("10.0.0.0/8");
    const b: Prefix4 = .parseComptime("10.1.0.0/16");
    const c: Prefix4 = .parseComptime("11.0.0.0/8");
    try std.testing.expect(a.overlaps(b));
    try std.testing.expect(b.overlaps(a));
    try std.testing.expect(!a.overlaps(c));
}

test "built from parts" {
    const subnet: Prefix4 = .init(.init(.{ 10, 1, 0, 0 }), 20);
    try std.testing.expectEqual(Prefix4.parseComptime("10.1.0.0/20"), subnet);
}

// A /32 is one host. Config that accepts either spelling can tell them
// apart without comparing bits.
test "a single host" {
    try std.testing.expect(Prefix4.parseComptime("10.1.2.3/32").isSingleIp());
    try std.testing.expect(!Prefix4.parseComptime("10.1.2.0/24").isSingleIp());
}

test "an address names its own prefix" {
    const addr: Ip4 = .parseComptime("10.1.2.3");
    try std.testing.expectEqual(Prefix4.parseComptime("10.1.2.0/24"), try addr.prefix(24));
    try std.testing.expectError(error.BitsOutOfRange, addr.prefix(33));
}

test "a prefix is parsed as strictly as an address" {
    try std.testing.expectError(error.NoSlash, Prefix4.parse("10.0.0.0"));
    try std.testing.expectError(error.BitsOutOfRange, Prefix4.parse("10.0.0.0/33"));
    try std.testing.expectError(error.InvalidBits, Prefix4.parse("10.0.0.0/08"));
    try std.testing.expectError(error.InvalidBits, Prefix4.parse("10.0.0.0/"));
}

test "a prefix formats with {f}" {
    const subnet: Prefix4 = .parseComptime("10.1.0.0/20");
    var text: [18]u8 = undefined;
    const formatted = try std.fmt.bufPrint(&text, "{f}", .{subnet});
    try std.testing.expectEqualStrings("10.1.0.0/20", formatted);
}

Convert from std types

Ip4.fromAny takes a std IpAddress and returns an Ip4, or null for a real IPv6 address. It also unwraps an IPv4-mapped IPv6 address. A dual-stack listener reports an IPv4 peer in that form, so fromAny gives one answer for both.

withPort makes an Ip4Address. fromIp4 drops the port. fromSlice builds an address from exactly four bytes. Ip4 formats with {f}.

const std = @import("std");
const net = @import("stdx").net;
const Ip4 = net.Ip4;

// A dual-stack listener reports an IPv4 peer as an IPv4-mapped IPv6
// address. `fromAny` gives the same `Ip4` for both spellings.
test "fromAny sees through an IPv4-mapped address" {
    const ip: Ip4 = .parseComptime("10.1.2.3");
    const native: net.IpAddress = .{ .ip4 = ip.withPort(5432) };
    const mapped: net.IpAddress = .{ .ip6 = .fromIp4(ip.withPort(5432)) };

    try std.testing.expectEqual(ip, Ip4.fromAny(native).?);
    try std.testing.expectEqual(ip, Ip4.fromAny(mapped).?);

    // A real IPv6 address has no IPv4 form.
    try std.testing.expectEqual(null, Ip4.fromAny(.{ .ip6 = .loopback(0) }));
}

test "an address and a port make a socket address" {
    const ip: Ip4 = .parseComptime("10.1.2.3");
    const socket_addr = ip.withPort(8080);
    try std.testing.expectEqual(8080, socket_addr.port);
    try std.testing.expectEqual(ip, Ip4.fromIp4(socket_addr));
}

test "fromSlice needs exactly four bytes" {
    try std.testing.expectEqual(Ip4.parseComptime("10.1.2.3"), Ip4.fromSlice(&.{ 10, 1, 2, 3 }));
    try std.testing.expectEqual(null, Ip4.fromSlice(&.{ 10, 1, 2 }));
}

test "an address formats with {f}" {
    const ip: Ip4 = .parseComptime("10.1.2.3");
    var text: [15]u8 = undefined;
    try std.testing.expectEqualStrings("10.1.2.3", try std.fmt.bufPrint(&text, "{f}", .{ip}));
}

Use an address as a key

An Ip4 is four bytes with no padding. It works as a key in the hash containers without a custom context.

const std = @import("std");
const stdx = @import("stdx");
const bounded_array_hash_map = stdx.bounded_array_hash_map;
const Ip4 = stdx.net.Ip4;

// An `Ip4` is four bytes with no padding, so it works as a key as it is.
const Hits = bounded_array_hash_map.Auto(Ip4, u32, 16);

fn record(hits: *Hits, client: Ip4) !u32 {
    const entry = try hits.getOrPut(client);
    if (!entry.found_existing) entry.value_ptr.* = 0;
    entry.value_ptr.* += 1;
    return entry.value_ptr.*;
}

test "count requests per client" {
    var hits: Hits = .init;
    const a: Ip4 = .parseComptime("10.0.0.1");
    const b: Ip4 = .parseComptime("10.0.0.2");

    try std.testing.expectEqual(1, try record(&hits, a));
    try std.testing.expectEqual(2, try record(&hits, a));
    try std.testing.expectEqual(1, try record(&hits, b));
}

Carry an address and a port

net.Ip4Endpoint is four octets and a port in 6 bytes: the value shape without the sockaddr padding that std's Ip4Address carries. Use it in per-connection records and in code that owns a raw fd and calls bind(2) or connect(2) directly. toSockaddrIn returns the kernel struct with the port in network order. parse takes the address text and the port as separate arguments, because a host:port split is caller policy. toStd and fromStd convert to and from Ip4Address.

Work with a raw socket fd

posix.setsockopt and posix.getsockopt set and read a one-c_int socket option on a raw fd. Both decode errno to named errors. error.InvalidOptionValue is EINVAL. std's own posix.setsockopt maps EINVAL to unreachable, so a bad option value from configuration has no error path. The stdx wrapper returns error.InvalidOptionValue instead. posix.getsockname and posix.getpeername return the local and peer net.Ip4Address of a socket, with no caller-supplied sockaddr buffer.

Know the limits

These types are IPv4 only. Ip4.parse rejects IPv6 text, and Prefix4.parse rejects IPv6 prefixes. For dual-stack matching, keep a second set of rules for IPv6 and test both. Socket calls on raw file descriptors are in posix.

UUIDs

Uuid generates version 4 and version 7 values, formats them, and parses the canonical 36-byte text form. Version 7 values are ordered across milliseconds, but not inside one millisecond.

const std = @import("std");
const Io = std.Io;
const Uuid = @import("stdx").Uuid;

test "generate, format, parse" {
    const id: Uuid = .v4();
    try std.testing.expectEqual(.v4, id.version());

    // The canonical form is 36 bytes.
    var text: [36]u8 = undefined;
    var writer: Io.Writer = .fixed(&text);
    try id.format(&writer);

    const parsed: Uuid = try .parse(&text);
    try std.testing.expect(id.eql(parsed));
}

test "a wrong length is an error" {
    try std.testing.expectError(error.InvalidLength, Uuid.parse("not-a-uuid"));
}

Timekeeping

The time module reads clocks without an Io value. It returns std.Io.Timestamp values, so the std Duration conversions work on the result. It runs on Linux and macOS.

Choose a clock

time.now takes an Io.Clock. Two clocks cover most code.

Clock	Use it for
`.awake`	Elapsed time, deadlines, and timeouts. The value has meaning only inside the process.
`.real`	Timestamps that people read, or that leave the process.

Never send an .awake value to another process. It is a number with no meaning there.

Measure a deadline

A deadline must read the clock directly. A cached sample can lag by one tick, and a timeout must not fire late.

const std = @import("std");
const Io = std.Io;
const timex = @import("stdx").time;

// A deadline reads the clock directly. A cached sample can lag by
// one tick, and a timeout must not fire late.
fn expired(started: Io.Timestamp, timeout_ms: i64) bool {
    const elapsed = started.durationTo(timex.now(.awake));
    return elapsed.toMilliseconds() >= timeout_ms;
}

test "a zero timeout is already expired" {
    try std.testing.expect(expired(timex.now(.awake), 0));
}

test "a long timeout is not" {
    try std.testing.expect(!expired(timex.now(.awake), 60_000));
}

Cache the clock in a loop

An event loop reads the time many times in each iteration. Each read is a system call. A CachedClock replaces all of them with one call at the top of the iteration.

const std = @import("std");
const timex = @import("stdx").time;

// One cache per thread. The thread that ticks it owns it.
threadlocal var clock: timex.CachedClock(.awake) = .init;

// The top of every loop iteration: one real clock read.
fn tick() void {
    _ = clock.refresh();
}

// Everything else in the iteration reads the cache.
fn stamp() i96 {
    return clock.now().nanoseconds;
}

test "every read inside a tick is the same sample" {
    defer clock.clear();
    tick();
    try std.testing.expectEqual(stamp(), stamp());
}

Follow these rules:

Give each thread its own cache. A CachedClock is not thread-safe.
Call refresh once per iteration. Call now for every other read.
A cached value is never in the future. It lags true time by at most one iteration.
A clock that was never refreshed reads the real clock directly.

Use the cache for log timestamps, metrics, and rate limits. Do not use it for deadlines.

Stamp log lines with wall time

A log line needs wall time, but the loop already holds an .awake sample. WallFromAwake converts the sample. It stores the offset between the two clocks, which the process computes once at startup.

const std = @import("std");
const Io = std.Io;
const timex = @import("stdx").time;

// The offset between the two clocks, computed once and shared.
var wall_from_awake: timex.WallFromAwake = .uninitialized;
threadlocal var clock: timex.CachedClock(.awake) = .init;

// Call once, before any thread starts.
pub fn init() void {
    wall_from_awake = .compute();
}

// A log timestamp: the tick's sample, converted to wall time.
pub fn logTimestamp() Io.Timestamp {
    return wall_from_awake.toWall(clock.now());
}

test "log timestamps are wall time and never go backwards" {
    init();
    defer clock.clear();

    _ = clock.refresh();
    const first = logTimestamp();
    try std.testing.expect(first.toSeconds() > 1_577_836_800); // after 2020

    _ = clock.refresh();
    try std.testing.expect(logTimestamp().nanoseconds >= first.nanoseconds);
}

The converted time never goes backwards. A clock step from NTP cannot reorder log lines. The cost is that the converted time drifts from true wall time if the system clock steps after startup. For log lines, this is the right trade. Use .real when you need true wall time.

Limit a repeated log line

A failure that repeats for each connection can write one line per event. RateLimit admits one line per interval. It returns the number of lines that it held back, so the one line can report them.

const std = @import("std");
const Io = std.Io;
const RateLimit = @import("stdx").time.RateLimit;

// A failure that repeats per connection logs once per interval. The
// line carries the number of failures it stands for.
fn logAcceptFailure(limit: *RateLimit, now: Io.Timestamp, out: *Io.Writer) !void {
    const held_back = limit.admit(now) orelse return;
    try out.print("accept failed, {d} more held back\n", .{held_back});
}

fn atMs(ms: i64) Io.Timestamp {
    return .fromNanoseconds(@as(i96, ms) * std.time.ns_per_ms);
}

test "a storm logs once per interval" {
    var limit: RateLimit = .default;
    var buffer: [256]u8 = undefined;
    var out: Io.Writer = .fixed(&buffer);

    // The first failure goes out. Nine more inside the interval do not.
    try logAcceptFailure(&limit, atMs(0), &out);
    for (1..10) |i| try logAcceptFailure(&limit, atMs(@as(i64, @intCast(i)) * 100), &out);
    try std.testing.expectEqualStrings("accept failed, 0 more held back\n", out.buffered());

    // The next interval logs again and reports the nine.
    try logAcceptFailure(&limit, atMs(1000), &out);
    try std.testing.expectEqualStrings(
        "accept failed, 0 more held back\naccept failed, 9 more held back\n",
        out.buffered(),
    );
}

// `default` is one second. `init` takes any `Io.Duration`.
test "a custom interval" {
    var limit: RateLimit = .init(.fromMilliseconds(250));
    try std.testing.expectEqual(0, limit.admit(atMs(0)));
    try std.testing.expectEqual(null, limit.admit(atMs(100)));
    try std.testing.expectEqual(1, limit.admit(atMs(250)));
}

admit returns null when the caller must hold the line back. Pass it a monotonic sample, such as clock.now() from a CachedClock(.awake).

RateLimit.default has a one-second interval. RateLimit.init takes any Io.Duration.

Timers

One operation does not need a set: every std wait already takes an Io.Timeout, and a .deadline in that value is the whole story. Use timers.Timers when one loop owns many deadlines. Each entry carries a calflback, removal is by handle, and a period repeats the fire.

The collection owns no clock and no wake machinery. The loop samples the clock with timex.now(.awake), calls expire(now) to fire everything due, and sleeps until next(). The entries are caller-owned: the caller constructs a Timer, standalone or embedded in a connection record, and arm links it. Nothing is allocated, and the node pointer is the handle. One loop owns the set: no method is thread-safe, and no atomics are involved.

Drive the loop

expire(now) fires every timer due at or before now, earliest first. The loop then sleeps toward next() with Io.Timeout{ .deadline = wake }. A null next() means nothing is armed, and the loop sleeps on its other events. A callback runs after the tree surgery on its own entry, so it can arm, reset, or stop any timer, including its own. After every callback, expire re-checks the minimum, so a past-due timer armed from inside a callback fires in the same drain.

stop and reset follow Go's Timer: each returns false for a timer that already fired or was stopped. Nothing is delivered outside expire, so there is no channel to drain and no stale delivery to reason about.

const std = @import("std");
const Io = std.Io;
const timers = @import("stdx").timers;

// The loop shape: expire what is due, then sleep toward next(). This
// example advances a simulated clock instead of sleeping.
test "a loop drains the timers that are due" {
    var set: timers.Timers = .empty;
    var connect: timers.Timer = .one_shot;
    var request: timers.Timer = .one_shot;
    var canceled: timers.Timer = .one_shot;

    set.arm(&connect, .fromNanoseconds(100), timers.disarm);
    set.arm(&request, .fromNanoseconds(200), timers.disarm);
    set.arm(&canceled, .fromNanoseconds(50), timers.disarm);

    // stop unlinks an armed timer and reports that it was armed.
    try std.testing.expect(set.stop(&canceled));
    try std.testing.expect(!set.stop(&canceled));

    // next() is the earliest armed deadline: what the loop sleeps
    // toward.
    try std.testing.expectEqual(Io.Timestamp.fromNanoseconds(100), set.next().?);

    var wakes: u32 = 0;
    while (set.next()) |wake| {
        wakes += 1;
        set.expire(wake);
    }
    try std.testing.expectEqual(2, wakes);
    try std.testing.expect(!connect.armed);
    try std.testing.expect(!request.armed);

    // Both fired: reset re-arms the entry and reports that it had
    // already fired.
    try std.testing.expect(!set.reset(&request, .fromNanoseconds(300)));
    try std.testing.expectEqual(Io.Timestamp.fromNanoseconds(300), set.next().?);
}

// A period timer: the callback returns .rearm until the tick count,
// then .disarm. The record embeds the Timer in a field named `timer`,
// so the typed callback receives the record back.
test "a period timer stops itself after three ticks" {
    const Ticker = struct {
        const Self = @This();

        timer: timers.Timer = .one_shot,
        ticks: u32 = 0,

        fn onTick(self: *Self, _: *timers.Timers, _: *timers.Timer) timers.Action {
            self.ticks += 1;
            return if (self.ticks < 3) .rearm else .disarm;
        }
    };

    var set: timers.Timers = .empty;
    var ticker: Ticker = .{ .timer = timers.Timer.repeat(.fromNanoseconds(10)) };
    set.arm(&ticker.timer, .fromNanoseconds(100), Ticker.onTick);

    // The re-arm anchors to the fired deadline: expiring late fires
    // only the ticks that are due.
    set.expire(.fromNanoseconds(105));
    set.expire(.fromNanoseconds(1000));
    try std.testing.expectEqual(3, ticker.ticks);
    try std.testing.expect(!ticker.timer.armed);
    try std.testing.expectEqual(@as(?Io.Timestamp, null), set.next());
}

Repeat on a period

A Timer with a period is Go's Ticker: it re-arms on every fire. The next deadline is the fired deadline plus the period, never the wake time. A late expire does not shift the schedule, so drift does not accumulate.

The callback decides the next fire: return .rearm to keep the timer going, or .disarm to let it die from inside its own fire. The callback runs between the fire and the re-arm, so it sees the entry disarmed and the decision pending. A callback that re-arms its own timer wins over its returned .rearm.

Recover the record

A callback is typed. Its first parameter is a pointer to the record that embeds the Timer. The wrapper recovers the record with @fieldParentPtr, so the record's field must be named timer. A void first parameter arms a bare timer. Timer.disarm is the ready-made callback for a timer whose fire is observed through armed and next().

The wrapper inlines the typed body into the erased callback, so a fire costs one function-pointer indirection. A callback that frees its own record must return .disarm.

Logging transport

A worker thread that writes a log line cannot block on a slow consumer. slot_ring.Mpsc is a bounded, lock-free ring for that shape: many threads acquire a slot, write, and commit, and one consumer drains in order. Producers never block: a refused acquire returns null and the new message drops. The module is Linux only: it is void on other targets. slot_size is a comptime option, and it must be a multiple of the cache line. init rounds the slot count up to a power of two.

Pick the wake flavor

The consumer's shape decides the wake mechanism at compile time.

Flavor	Consumer
`.none`	The caller drives every drain: a test, or a loop that drains on its own schedule.
`.eventfd`	A poll loop. `pollFd()` is the doorbell. Call `ackDoorbell()` when the fd reads ready, before the drain.
`.futex`	A dedicated consumer thread. `waitForData` parks it until data or shutdown. `requestShutdown` ends it after the producers join.

Pick the loss policy

.drop_new is the default and the logging choice. A full ring refuses the new message, so the loss is the newest lines. .drop_oldest displaces the oldest committed message instead, so the newest line wins. The displacing ring owns the consumer cursor, and the consumer reads through read(buf), which copies the payload into the caller's buffer. A read(buf) that loses the cursor to a displacer returns null, and the displacer ticks displacedCount.

Drain in order

The consumer stops at the first null, which marks a claim that is not committed yet. A consumer that advances past the first uncommitted claim skips the committed messages behind it. Under .drop_new, the caller owns the read position and release frees the slot. Under .drop_oldest, the ring owns the cursor and read(buf) frees the slot inside the window it claims.

Account the loss

head() counts the claims handed out. A refused acquire is the caller's drop count. Under .drop_oldest, displacedCount() counts the lines that producers displaced. Every line is delivered, displaced, or refused, so the counts close the total.

const std = @import("std");
const slot_ring = @import("stdx").slot_ring;

// Four 256-byte slots, the default wake flavor and the default drop
// policy: nothing wakes the consumer, and a full ring refuses the new
// line. Worker threads claim, write, and commit; the caller drains.
const LogRing = slot_ring.Mpsc(.{ .slot_size = 256 });

const Log = struct {
    ring: *LogRing,
    // The single consumer owns the read position. It advances only
    // past a committed line that it released.
    read_pos: u64 = 0,
    dropped: u64 = 0,

    // The producer path: claim, write, commit. A refused claim counts
    // the drop and returns, so no path blocks.
    fn write(self: *Log, line: []const u8) void {
        var lease = self.ring.acquire() orelse {
            self.dropped += 1;
            return;
        };
        var w = lease.writer();
        w.writeAll(line[0..@min(line.len, LogRing.payload_size)]) catch unreachable;
        lease.commit(self.ring, @intCast(w.end));
    }

    // The consumer path: read in order from the owned position and
    // stop at the first uncommitted claim, so nothing behind it is
    // skipped.
    fn drain(self: *Log, out: *std.Io.Writer) !void {
        while (self.read_pos < self.ring.head()) : (self.read_pos += 1) {
            const msg = self.ring.read(self.read_pos) orelse break;
            try out.print("{s}\n", .{msg.payload});
            msg.release();
        }
    }
};

test "a full ring drops the newest lines and the drain stays in order" {
    var ring: LogRing = undefined;
    try ring.init(std.testing.allocator, undefined, 4);
    defer ring.deinit(std.testing.allocator);
    var log: Log = .{ .ring = &ring };

    // Six lines into four slots: the first four commit, the last two
    // are refused.
    for (0..6) |i| {
        var buf: [16]u8 = undefined;
        log.write(try std.fmt.bufPrint(&buf, "line {d}", .{i}));
    }
    try std.testing.expectEqual(4, ring.head());
    try std.testing.expectEqual(2, log.dropped);
    // Every line is committed or counted: nothing vanishes.
    try std.testing.expectEqual(6, ring.head() + log.dropped);

    var buffer: [128]u8 = undefined;
    var out: std.Io.Writer = .fixed(&buffer);
    try log.drain(&out);
    try std.testing.expectEqualStrings("line 0\nline 1\nline 2\nline 3\n", out.buffered());

    // The drain released every slot. The next line lands, and the
    // consumer resumes at its own position.
    log.write("after");
    try std.testing.expectEqual(5, ring.head());
    var more: [32]u8 = undefined;
    var out2: std.Io.Writer = .fixed(&more);
    try log.drain(&out2);
    try std.testing.expectEqualStrings("after\n", out2.buffered());
    try std.testing.expectEqual(5, log.read_pos);
}

Tradeoffs

The ring is not a lossless queue. Overflow drops a message under either policy: .drop_new refuses the newest, and .drop_oldest displaces the oldest committed message when it can. The loss is always accounted (head(), a refused acquire, displacedCount()) but never prevented. A caller that must not lose a message needs a blocking channel, not this ring.

One consumer thread drains by position, in order. A producer stalled mid-write stalls the drain behind its position: the cost is dropped messages, never corrupted ones. There is no out-of-order drain and no parallel drain.

Producers never block. An acquire either wins a slot or returns null, and the caller drops the message. Under .drop_oldest, a full ring displaces the oldest committed message instead, so the newest message wins. A stalled producer still refuses the claim, and so does a displacement that cannot win the cursor.

The policy decides the read shape. Under .drop_oldest, the consumer reads through read(buf) and gets a copy inside the won cursor window. .drop_new hands out a borrowed slice from read(read_pos): the single consumer owns the position, and nothing can take the slot before release.

One message costs one CAS on the head, the payload copy into the slot, and one release store of the stamp. The wake flavor adds its own cost. .none adds nothing, .eventfd writes eight bytes to the fd, and .futex bumps the wake word with one RMW. The .futex syscall runs only when a waiter is armed. A displacement adds a cursor CAS, a stamp CAS, and one atomic increment of the displaced count. It also wakes the consumer under the two waking flavors.

Linux only. The .eventfd and .futex flavors wrap Linux syscalls, and the module is void on other targets. slot_size is comptime and a cache-line multiple. Changing it instantiates a new ring type.