Wednesday, November 18, 2026

FLStore: An Encrypted Storage Engine for Delphi!

SQLite is the default answer whenever a Delphi application needs local storage, and for good reason. But many applications never actually need SQL. They store records, find them again by key or by index, and on mobile devices they need to keep that data encrypted. For exactly this job we built FLStore: a page-based CRUD storage engine that plugs into our TVectorDB interface, with encryption built in. This post explains how the engine works, how it performs against SQLite (with and without encryption), and how we tested it.




One Interface, Three Drivers

TVectorDB is our table-oriented data layer. The model is a plain Delphi record with attributes; TFLVectorTable<T> reads and writes it through the IFLTableStore interface. Behind that interface there are three drivers. The MySQL and SQLite drivers generate SQL. The third one, FLStore (TStorageTableStore), is a storage engine of our own with no SQL anywhere. The application code is the same for all three.

The model attributes and nullable types come from our FDK. A minimal example:

type
  [VectorTable('customers')]
  [VectorIndex('City')]
  TCustomer = record
    [DBField(true, 32)] GUID : string;
    [DBField(26)]       Name : string;
    [DBField(29)]       City : string;
  end;

var
  lStore    : IFLTableStore;
  lCustomer : TCustomer;
  lTx       : IFLVectorTransaction;
begin
  lStore := TStorageTableStore.Open('customers.fls');  // encrypted: Open(FileName, KeyMaterial)
  lStore.EnsureTable(TFLModel<TCustomer>.Schema);

  lCustomer      := Default(TCustomer);
  lCustomer.GUID := 'G1';
  lCustomer.Name := 'Miller';
  lCustomer.City := 'Boston';
  TFLVectorTable<TCustomer>.Construct(lStore).Create(lCustomer);  // autocommit

  lCustomer := TFLVectorTable<TCustomer>.Construct(lStore).Read('G1');

  lTx := lStore.BeginTransaction;
  TFLVectorTable<TCustomer>.Construct(lStore).Update(lCustomer);
  lTx.Commit;

  // Queries: TFLVectorTable<TCustomer>.Construct(lStore).Query.OrderBy('GUID').ToArray
end;

The same code runs unchanged against the SQLite or MySQL driver. Moving from SQLite to FLStore means changing a single line: the store you open.


Why Drop the SQL Text Layer?

With an SQL driver, every single operation takes a detour. The driver builds an SQL string, the database parses it and plans it, parameters are bound, and the result columns are converted back into record fields. For an application that only stores and retrieves records, all of that is overhead. In FLStore the record passes through a field table that is registered once and goes directly into the row format. Searches are structured conditions, not text.

How little overhead remains shows up in a sampling profile of a single insert with commit (Win64). Roughly three quarters to four fifths of the time is spent in the one WriteFile call that appends the commit to the journal. The engine code itself accounts for about 5%. What is left is essentially the cost of the storage hardware.


Under the Hood

  • File layout: one main file, two alternating journals and a double-write area. Data pages are 16 KB and are grouped into segments of 64 pages. B+-tree index pages live in a shared index region at the front of the file.
  • Redundant metadata: two header slots and two copies of the segment table. On open, the valid header with the higher sequence number wins.
  • One file format everywhere: little endian with fixed sizes, so the same file works on Win32, Win64, Android64 and iOS64.
  • Schema evolution for free: each value in a row is stored with its column id and type. A new column gets the next id, and old rows read it as NULL. ADD COLUMN never rewrites a single row.
  • Indexes: a primary-key tree plus secondary indexes ([VectorIndex]), composite ones included. They support equality on the leading columns, ranges and "starts with" on the next one, IN lists, and ORDER BY straight from the index order.
  • Commits: a commit appends the changed pages to the active journal as delta frames (only the bytes that changed) in one write call. A background thread syncs the journal once per second and copies a full journal into the main file (checkpoint). Because the two journals alternate, a commit never waits for a running checkpoint.
  • Crash safety: a killed process loses nothing. A power loss can cost at most roughly the last second, and in both cases the file stays consistent. The double-write area protects against torn pages during a checkpoint.
  • Threading: one instance per file and process, and every method runs under the instance lock. A transaction belongs to its thread, and other threads never see uncommitted rows.

Encryption Built In, Not Bolted On

Encryption in FLStore is not a layer on top. It is part of the page format. A page is encrypted wherever it leaves memory: in the main file, in the journal including its delta payloads, and in the double-write area.

  • A key per file: every file has its own random 256-bit data key, stored wrapped in the file header. A check value detects a wrong key the moment the file is opened.
  • Crash-safe by design: every page version is encrypted with its own unique parameters, and these are never reused, not even across crashes.
  • Changing the key costs almost nothing: only the data key in the header is rewrapped, and no page is rewritten. Re-keying a large file takes the same time as re-keying an empty one.
  • Only what is read gets decrypted: the engine decrypts in small blocks and touches only the bytes a read actually needs, instead of decrypting whole 16 KB pages.

The Test Setup

All numbers were measured on 18 November 2026 on a real, mid-range Android phone, not on a developer workstation. All configurations run in the same benchmark app, from a single build, through the same TVectorDB layer and on the same data:

ParameterSQLiteFLStore
DeviceSamsung Galaxy A15 (SM-A155F), Android 16 / API 36
BuildDelphi 13.2, Android64, Release
Data140,000 rows; Int64 key, 52 text columns of 48 characters each, two secondary indexes; plus a small table for single inserts
EngineSQLite 3.42.0 (FireDAC, statically linked)FLStore, current version
Encryptionnonemeasured both off and on
Durabilityjournal_mode=WAL, synchronous=NORMAL: no fsync per commitno fsync per commit, journal synced about once per second
Cache32 MB32 MB
Runsmean of 4 runsmean of 2 runs per mode (order: off, on, on, off)

Each run has three rounds, and the reported value is the median per operation. Both engines are configured for the same durability class: consistent after every crash, with neither forcing an fsync on every commit. The details differ: FLStore limits the window of possible loss to about one second, while SQLite syncs its WAL at checkpoints. SQLite has no built-in encryption, so it appears only unencrypted.


The Numbers

Time per operation in µs (lower is better). The factor in brackets is SQLite divided by FLStore, so a higher factor means FLStore is faster:

OperationSQLite (unencrypted)FLStore (unencrypted)FLStore (encrypted)
Batch insert (1,000 per transaction), per row153.747.5 (3.2x)65.9 (2.3x)
Read by key283.263.1 (4.5x)81.1 (3.5x)
Range of 100 rows5,7293,956 (1.4x)4,668 (1.2x)
Equality via secondary index2,4142,015 (1.2x)2,130 (1.1x)
Full scan, per row45.325.0 (1.8x)31.8 (1.4x)
Single update with commit (total)1,199.2182.8 (6.6x)314.7 (3.8x)
  – median commit619.6113.0 (5.5x)210.6 (2.9x)
  – p99 commit1,901.4186.0 (10.2x)385.8 (4.9x)
Single insert with commit (total)138.023.3 (5.9x)27.6 (5.0x)
  – median commit100.421.8 (4.6x)25.1 (4.0x)
  – p99 commit292.559.4 (4.9x)74.7 (3.9x)
Open database1,087499 (2.2x)634 (1.7x)
File size after close178.7 MB174.3 MB174.3 MB

Unencrypted against unencrypted, FLStore is faster in every case, between 1.2x and 6.6x, and up to 10x on the p99 commit. Even with every page encrypted, FLStore stays ahead of unencrypted SQLite in every case, between 1.1x and 5.0x. The biggest advantage shows up where interactive applications actually spend their time: small transactions with their own commit. The tail latency matters most for a responsive UI. Even encrypted, the p99 commit of a single update is 0.39 ms with FLStore, compared with 1.9 ms for SQLite.

The slowest single commit per round (at a journal switch or checkpoint) is a one-off event that varies a lot from run to run. It is therefore not listed in the table; median and p99 are the reliable figures. For the record: the slowest single update commit stayed below 9 ms in all FLStore runs, against roughly 80 to 90 ms for SQLite. The slowest single insert commit was 0.25 ms unencrypted and 0.36 ms encrypted, against 30 to 41 ms for SQLite.

What Encryption Costs

Comparing the two FLStore columns shows the price of encryption directly. It is highest on writes: the median commit of a single update goes from 113 µs to 211 µs, and batch inserts from 48 µs to 66 µs per row. On reads it is under 30% (63 µs to 81 µs per read by key), thanks to decrypting only what is actually read. The file size is identical with and without encryption.

For Reference: SQLite With Default Settings

The table above uses a tuned SQLite (synchronous=NORMAL, 32 MB cache), because that puts it in the same durability class as FLStore. An application that simply opens SQLite with the defaults of the FireDAC build gets different commit times. Same device, same benchmark, two further runs:

Operation (µs)SQLite, defaultsSQLite, WAL + NORMAL + 32 MB
Batch insert, per row157.5153.7
Read by key281.2283.2
Full scan, per row45.245.3
Single update, median commit3,319.9619.6
Single update, p99 commit4,523.91,901.4
Single insert, median commit421.4100.4

Reads, scans and batch inserts are practically unchanged (within ±2%), so the larger cache brings nothing measurable at this data size. Single commits, however, are 4.2x to 5.4x slower at the median with the defaults. The timings are consistent with an fsync on every commit, which is a stricter durability class than FLStore's. These numbers show what an application gets out of the box, but they are not an equivalent comparison. That is why our headline figures use the tuned configuration.


How We Tested: The Part That Matters Most

Writing your own storage engine is easy to claim and hard to trust. That is why the speed figures are the smaller half of this post. A database is only worth using if it gives back exactly what was committed, after every kind of failure. Here is what stands behind FLStore.

1. Correctness first: 575 engine tests, green on Win32 and Win64. The DUnitX suite of the storage engine covers the file format, header A/B fallback, damaged tables and pages (CRC errors, wrong page numbers), journal scans with a torn or damaged tail, open transactions, recovery after a killed process, double-write, the background checkpoint, encryption (including known-answer tests against the published reference vectors), backup and restore, and multiple threads and handles on one file. All 575 tests pass on both Win32 and Win64.

2. Fault injection: a crash at every single write. We don't test "a crash somewhere". Dedicated tests simulate a crash at every write operation: the simulated crash writes only part of the data and aborts, the file is reopened, and the test checks that exactly one committed state is present. This runs for the commit path, the background checkpoint and the first write after open, both unencrypted and encrypted.

3. Red-green proof for every fix. A test that passes proves nothing if it would also have passed without the fix. For every bug fix, our tooling (delphi__red_green_check) reverts the production change in a throwaway copy, keeps the new test and runs the build. The test must turn red without the fix and green with it. Otherwise it does not test what it claims to test.

4. Tested in a real application. The first production application on FLStore brings its own suite of 925 DUnitX tests, a large share of which exercise its repositories on top of the engine. They check that IDs are never handed out twice (not even after a Clear), that deletes really delete, that key changes work, that NULL stays distinct from 0.00, and that everything survives a reopen.

5. A fair benchmark, by construction. The benchmark app (an FMX program) measures all configurations in the same process, one after the other on a background thread. The order alternates every round, so no configuration always gets the cooler device. The input is identical for all: a fixed random seed and a prebuilt text pool. Timing uses TStopwatch, and the report gives the median per operation, plus median, p99 and maximum of the individual commits for the commit cases. Every round starts on a fresh file, the screen is kept on, and partial results are written after each round so that an aborted run loses nothing.

6. SQLite as the control group. SQLite runs in every pass. Because its code never changes, it serves as a control: if its numbers drift between runs, the device is the cause and the run is not valid. Engine optimizations are verified in A–B–B–A order with a fresh install for every run, so that a warm or cool device cannot fake a gain.

7. Verify the result, not just the clock. After reopening, the benchmark checks that the file really contains all rows. If anything does not add up, the run aborts instead of printing an attractive number. A benchmark that stops is worth more than one that flatters.

8. Clean conditions or no numbers. Measurements are only accepted on an idle machine, and the background load is logged for every run. Measurements taken under load are discarded and repeated. Each optimization was found with a sampling profiler (one sample every 250 µs, resolved through the map file) and then measured on its own.

9. Long enough to see the truth. Effects that involve the background checkpoint only show up in long runs. 10,000 single updates were not enough to make one optimization visible; with 100,000 updates it was clear (168 → 106 µs per update). Short benchmarks can hide exactly the behaviour that matters in production.


Tooling

FLStore was developed with our own Delphi infrastructure: the FDK for the model attributes, nullable types and base classes, and a backend with 33 MCP servers. DelphiMCP builds every target (Win32, Win64, Android64, plus the benchmark app against any engine version), runs the unit tests and performs the red-green check. PlanMCP (MySQL-backed) keeps tasks and decisions, and hgMCP handles version control. Development and on-device measurement ran in parallel and were coordinated agent-to-agent. A complete benchmark series on real hardware takes about an hour, with no manual build or deployment steps.


Conclusion

If an application stores records rather than running ad-hoc SQL, the SQL text layer is a cost it pays on every call. FLStore removes that layer. In the same durability class it is up to 6.6x faster than SQLite, and with every page encrypted it is still up to 5.0x faster than unencrypted SQLite, with slightly smaller files and a much shorter commit tail. Because it sits behind the TVectorDB interface, switching is a one-line change.

Numbers like these are only worth something if you can trust how they were made. Hundreds of tests, a simulated crash at every write, red-green proof for every fix, alternating run orders with SQLite as the control group, and clean measuring conditions: that is the foundation FLStore stands on.


Note on the Use of AI: This article was created with the "assistance" of generative AI. The content, technical statements, and conclusions have been reviewed and revised by the author. The author bears full responsibility for the publication.


Thursday, October 22, 2026

ISOlDE Gameengine: What an Experienced Developer Can Achieve in Three Days With AI

A game engine with its own IDE, a sprite editor, a shader graph and hot reload — all in Delphi, all in a few days. This post is about the engine, but even more about what makes such speed possible: frameworks, a backend of cooperating MCP servers, and a plan database that never forgets.

Meet ISOlDE — a Game Engine Written in Delphi

On 25 September 2026 I started a new repository. On 28 September, version 0.5.0 of ISOlDE was there: an isometric 2.5D game engine with its own IDE, both written entirely in Delphi. Direct3D 11 renderer, sprites with per-pixel depth images for the terrain, meshes with GPU skinning for characters and objects, dynamic light with a time of day, weather, fire, smoke, water and fog. And the game code itself is Delphi: the IDE compiles it into a DLL and swaps it into the running game every time I press save.

The whole thing is not built on a foreign engine and not on a foreign scripting language. It stands on frameworks I have developed over the years, and that is exactly why it could grow so fast.

The Foundation: FDK, #D.MVVM and the FMX Code Editor

  • FDK — my Delphi framework with well over 100 units. It carries everything an application needs below the user interface, and the engine and the IDE both use it. More on the FDK.
  • #D.MVVM — the MVVM framework for Delphi. The IDE with its docking windows, panels and plugins is built on it, so every tool window has a clean view model that can be tested without a user interface. The story of D.MVVM.
  • FMX code editor — my own FireMonkey implementation of a source code editor in the style of SynEdit. The game code is edited right inside ISOlDE, and a press on save triggers compile and hot reload.
  • FMX ribbon controls — the ribbon of the IDE, including tabs and customisable key bindings (Delphi, CUA or empty preset).

Four building blocks, and every one of them already existed. An agent that gets a clear, well-tested framework does not have to invent a user interface layer, a docking system or an editor first. It uses them — and produces code that fits.

The Real Secret: 33 MCP Servers That Work Together

Several Claude instances worked on ISOlDE at the same time. That is only possible because of my backend server concept: today, 33 MCP servers interact with each other through one central backend. A proxy in front of it hands every agent exactly the tools it needs — compiler, source index, tests, search, Word, mail, Blogger, ffmpeg, and so on. Agents find each other, ask each other questions, wake each other up, and lock a unit when they are about to change it, so two of them never write into the same file.

Compiling does not happen through a hand-typed command line but through DelphiMCP; searching a code base of several hundred thousand lines goes through an index instead of a blind grep. This is the part that turns "an AI that types code" into "a team of agents that delivers". The modules are available as Delphi source code — see the MCP page.

I don't think I need to mention this specifically, do I? Of course, the game engine has its own MCP server so that agents can start working with it right away and create games with minimal effort. Many years ago, I programmed a Pac-Man* clone in FMX and had already fully implemented the game logic here. Claude rewrote the game in no time using the built-in MCP server in the backend.

*) - original © by Namco -

PlanMCP: Organisation in a Database, Not in Markdown Files

Most agent setups keep their plans in Markdown files that someone has to maintain, merge and re-read. Mine does not. PlanMCP stores tasks, features, decisions, and handovers in a MySQL server. Any agent can ask: What is open? What is blocked? What did the last session hand over? When I clear a session, the next one picks up exactly where the last one ended — with an append-only history, not with a text file that may be out of date.

For ISOlDE this meant 166 tracked features, each with a status, and a plan that all agents see in the same state at the same moment. With several agents working in parallel, that is not a nice-to-have. It is the difference between coordination and chaos.

A PAServer for Games: IDE in the VM, Game on the GPU

I work inside a VMware virtual machine, and a VM has no real GPU. A game engine needs one. So I took the idea behind Delphi's PAServer and applied it to games: the IDE runs in the VM, and a small runner runs on the Windows host, where the RTX 5090 sits. The runner listens in the local network and authenticates the IDE with a shared token. It idles without a Direct3D device and only starts rendering when it is told to run.

  • F9 builds and starts the game locally for debugging.
  • Shift+F9 builds the release, deploys it from the VM to the host (about 117–128 MB/s over the LAN), loads the game DLL and hot-reloads it — while the game keeps running.
  • The IDE even brings the host runner up to date by itself, and a health call reports its build and protocol version.
  • A command-line tool, IsoCtl, offers the same commands to scripts and agents: run, stop, stats, capture, deploy, update.

The result: the comfort of working in a VM and the speed of real hardware. On the host, a borderless 4K window with one million sprites runs at about 550 fps, and the volcano island scene at over 1,000 fps.

Screenshots and 4K/60 Demo Videos, Straight From the GPU

A game engine needs pictures. IsoCtl capture fetches the current frame from the host in full resolution (4K included), so agents and I always look at the real GPU image, not at a screen grab of the VM.

For videos, the runner records itself, and it does so properly. No screen capture, no wall clock: a fixed time step of 1/fps, vsync off, and every single frame is read directly from the GPU interface and handed to ffmpeg — with hardware encoding (NVENC for h264, hevc or av1) or libx264. The result is a demo video in 4K at 60 fps (the frame rate can be set from 1 to 240), rendered faster than it plays back.

The ffmpeg part is, of course, one more server in the backend: an ffmpeg MCP server installed on the GPU machine itself. That is where the frames are, so nothing has to travel over the network before it is encoded. An agent can record a scene, cut it and post-process it with a few commands, instead of me spending an evening with a screen recorder.

What Happened, Day by Day

DayRevisionsResult
25.09. (start)20New repository, resources imported, runner without Direct2D, monitor selection, full screen
26.09. (day 1)35IDE with docking, game DLL with hot reload, content store, scene designer with undo, FBX characters with GPU skinning
27.09. (day 2)108Shader graph with HLSL generation, flowing water and lava, sprite import, FBX models in scenes, plugin API, sprite editor as a plugin
28.09. (day 3)113Plugin host, sprite editor as a standalone application, depth images for sprites, dynamic light with time of day, version 0.5.0

276 revisions in four calendar days. On the last day, the volcano island demo already had wind-blown grass, bridges, flowing lava and smoke — once the tools exist, building a scene with them is the fun part.

Why It Goes This Fast

  • Parallel agents with a fixed contract. Each agent has its own files and a precisely described interface. The merge commits in the history show that several of them really were working at the same time.
  • Test first, then the core. Calculation kernels get their unit tests before the implementation, which gives the agent a solid feedback loop.
  • Experience that steers. 45 years of development decide the architecture, the order and the acceptance. The agents do the typing, the porting and the research across many files — and this is where the hours disappear.
  • Looking at the result. A short look at the screen finds what no test catches. An animated fox that kept rendering in the editor, a splash screen that waited too long — found, fixed, and turned into a rule or a test.

FBX to 2.5D Sprites, Sprite Editor and Mesh Editor

FBX import: my own FBX importer reads static models as well as animated characters. Whole model kits (Kenney, CC0) stand in the isometric scene as real 3D objects, lit together with the 2.5D sprite world, and an animated fox runs with GPU skinning of up to 256 bones. And the importer is also the way into the 2.5D world: FBX models are baked into 2.5D sprites with depth image and normals. That is how the bridges on the volcano island were made — a 3D model goes in, a sprite that sorts and lights correctly in the isometric scene comes out. Because every sprite carries its depth and normal channels, flat 2.5D sprites and real 3D meshes can live side by side in the same scene.

The sprite editor is a result of those first days: it paints in isometric planes, supports surfaces with normals for dynamic light, patterns, layers and directions, and it runs either as an IDE plugin or as a standalone SpriteEditor.exe. Shader effects such as torches, geysers and lava can be baked into looping sprite sheets.

The mesh editor followed shortly after on the same plugin API, and it bakes depth images from points and meshes into sprites. A nice proof that a clean plugin interface pays off — a new tool is a plugin away.

A tip from daily practice: the AI verifies against what it can see. When the mesh editor was checked through the CPU drawing path while I looked at the GPU view, we found and fixed a silent fallback on window resize — and now the screenshot function captures the real GPU views. Give the agent the same view the human has.

A Local Assistant for the Tedious Part

Not every job needs the biggest model. ISOlDE has a local agent panel running qwen3.8 on my own hardware. It takes over bulk work: assessing an imported tile set, suggesting roles and semantics, generating maps. Assigning roles to tiles worked 12 of 12 times; for edges and orientation geometry, code takes over, because code simply measures better than a model estimates. A larger model is used only for acceptance. Local model for volume, code for what can be calculated, big model for judgement.

ISOlDE Today

MeasureValue (22.10.2026, four weeks in)
Revisions1,333
Delphi source files847 files, about 622,000 lines
DUnitX tests3,574, all green, no leaks
Shader graphroughly 300 nodes
Frame rateover 2,000 fps on an RTX 5090 without optimisation (target: 400 fps as reserve)

What an Experienced Developer Can Achieve in Three Days

A lot. A running game engine with IDE, editors, FBX-to-sprite import, a host runner on the GPU, hot reload, and 4K/60 demo videos — because the pieces fit together: FDK and #D.MVVM as the foundation, the FMX editor and ribbon as the building blocks, 33 MCP servers as the workshop, and PlanMCP as the memory that keeps the whole team on the same page.

Delphi plus AI does not mean writing less. It means being able to build much more — as long as the framework and the infrastructure around the agents are right.

ISOlDE is in active development. Screenshots, the feature list, and a trial are on delphiprofi.de/isolde, and the frameworks and MCP modules behind it are on delphiprofi.de.


Note on the Use of AI: This article was created with the "assistance" of generative AI. The content, technical statements, and conclusions have been reviewed and revised by the author. The author bears full responsibility for the publication.



Saturday, October 10, 2026

When Using AI Stops Counting as Real Work

Have you noticed this too? No matter how good you are as a software developer, the moment you've used AI for even part of the work, your own work gets quietly devalued.

You sit for hours in front of the monitor, watching the terminal window where an agent is working, alert as a fox outside a rabbit hole, waiting for the exact moment the agent starts to go off the rails — so you can hit ESC before it does something you'll regret.

In between, the agent keeps asking questions, and you have to answer them as precisely as you possibly can. If you don't, five minutes and three units later you get the bill: the agent just assumed something, because it simply misunderstood you.

Often the biggest hurdle isn't that the question runs forty lines or more — it's that you have to ask a follow-up just to understand the question in the first place. Especially when a sub-agent has been working for an hour where you couldn't follow along, and suddenly there are facts in the world you have no idea about. It's one reason I actually prefer an agent to open a real terminal window for inter-agent communication, so I can read along there too.

Here's the flip side of the coin: the moment "the others" find out you use AI, it stops mattering from that point on whether you typed it yourself or an agent did. It no longer counts as real work.

I think a shift in thinking has to happen here.

Sure, the models keep getting better, and they can generate entire flows, games, or complex websites from a single long prompt. But there's a steep gradient in how good the models actually are -  proportional to how much material was available at training time.

Still, the result isn't uniform — at best it only mirrors the task you gave it. Much like handing a kid the instruction: clean your room, and once you're done you can go play. If it has to happen fast, everything ends up under the bed or stuffed in the closet, not actually put away.

It's no different with agents. We call it the feedback loop — a reward for the agent, or the lack of one, that it uses to judge its own work. In our case, that's unit tests, for example. If the task is "all unit tests must be green before a commit," the agent will happily take a shortcut built specifically so that exactly those unit tests turn green — which is a long way from proving the application actually does what we designed it to do.

A method an agent writes on its own is not the same as a method we describe precisely, with a fixed goal, and then let the agent type out. That's exactly where the difference lies.

Left to its own devices, the agent might reach for an array and search it with a for-loop. What we actually want is a linked list, a dictionary, or a B-tree. This is where the developer comes in. Only the one who steers the agent precisely gets what they actually wanted. This is exactly where the wheat separates from the chaff.

Can you see that from the outside? Not necessarily — and in a lot of places, performance simply doesn't matter these days anymore, sadly. Even if we no longer look at every single line of code, we know what we built by the end of the day, and that's ultimately what makes the difference.

The era of developers typing every line themselves is pretty much over, and the typing developer will eventually "go extinct" - even if plenty of people don't want to believe that yet. Especially in software development, the speed and the demands on new applications have risen so much that typists simply can't keep up over the long run.

There will certainly still be a need for a software engineer to look closely at core routines. But what percentage of the whole industry will that actually be?


Note on the Use of AI: This article was created with the "assistance" of generative AI. The content, technical statements, and conclusions have been reviewed and revised by the author. The author bears full responsibility for the publication.

Monday, September 28, 2026

Local LLMs for Delphi Migration — Follow-Up: Swapping the Inference Engine, Not the Model

This is a follow-up to the three-part series on running local LLMs against a structured, five-phase (AT1–AT5) Delphi migration benchmark, and to the follow-up comparing qwen3.6:35b-a3b against KAT-Coder-V2.5-Dev on the same weights class. This post holds the model constant and swaps the inference engine instead: same GGUF-class model, same benchmark harness, Ollama against a purpose-built CUDA engine called NInfer.



A correction up front, because it affects a number this series already published. The KAT-Coder follow-up stated that both qwen3.6:35b-a3b and KAT-Coder-V2.5-Dev held “100% GPU utilization with no CPU offloading” at NumCtx=131072. New validation data gathered for this post shows qwen3.6:35b-a3b running at ~91% GPU under that configuration on the current hardware — not the fully GPU-resident state previously reported. We're flagging this because the series' own standard is to publish a correction the moment a later measurement contradicts an earlier one rather than let a wrong number stand uncorrected.


What Changed: The Engine, Not the Model

Everything so far in this series ran on Ollama. This round adds NInfer, a C++/CUDA inference engine built specifically for the RTX 5090 (Windows forknatpate/ninfer-windows, portable zip, no install). It does not use GGUF — it has its own model format (.ninfer), and the model under test here is Qwen3.6-35B-A3B converted to that format at “groupwise-int” quantization (20.6 GiB on disk, KV-cache in bf16, 150k context). The Ollama side of the comparison is the same qwen3.6:35b-a3b Q4_K_M this series has used throughout.

The benchmark harness itself did not change to make this comparison possible: the existing run_benchmark.ps1 gained one additional code path, -Engine ninfer, which talks to NInfer's OpenAI-compatible API. The Ollama path is untouched. Same test cases, same prompts, same scoring functions. Settings held constant across both engines: NumCtx 131072 for AT1–AT3, 4096 for AT4, temperature 0, max 8192 output tokens, thinking enabled on both sides.

One asymmetry worth stating plainly: the Ollama scores in this comparison are the existing result files from earlier runs (AT1: 2026-05-20, AT2/AT3: 2026-08-03, AT4: 2026-05-18) — they were not re-measured for this post. Only the NInfer side is new. This is a deliberate reuse of already-validated numbers, not a re-run under identical wall-clock conditions; keep that in mind when reading the timing comparison below.


The Numbers: Ollama vs. NInfer, Same Model, Same Harness

Suite n (O/N) Ollama (all) NInfer (all) n (common) Ollama (common) NInfer (common) Unit
AT128/2841.654.62644.654.9Hit %
AT1-DE30/2838.542.62840.642.6Hit %
AT2101/11264.972.19369.974.4Hit %
AT330/3068.478.73068.478.7Structure+content %
AT490/900.8390.846900.8390.846Score 0–1
AT573/730.9961.000730.9961.000Score 0–1
AT660/600.7720.800600.7720.800Score 0–1

Why “n (common)” exists: NInfer's 150k context rejects two source files outright — one 453,190 characters, the other 463,994 characters — with an explicit context_length_exceeded error, costing it 10 tasks across AT1/AT2. Ollama does not reject the same prompts. It silently truncates them tonum_ctx: for all five AT1 tasks against the larger file, the returned prompt_tokens is exactly 131072 — the num_ctx value, not the file's actual token count. This means every prior AT-series result this blog has published for oversized files was scored against a truncated version of the source, not the file the reader would assume was analyzed. NInfer's hard rejection is, in this specific respect, the more honest failure mode. The “n (common)” columns above restrict both engines to tasks both could actually complete on the full, untruncated input — that is the fairer of the two comparisons in this table.

On quality: every suite improves or ties under NInfer, most visibly AT1 (+13 points) and AT3 (+10.3 points). AT4/AT5/AT6 — already near ceiling — move by a point or two, consistent with there being little room left to gain.


Throughput: The Actual Headline

SuiteOllama total wall timeNInfer total wall timeSpeedup
AT1912 s192 s4.75×
AT24,076 s977 s4.17×
AT3622 s137 s4.54×
AT4403 s99 s4.07×
AT5668 s53 s12.6×
AT6422 s94 s4.49×

And at the token level, mean (median) tokens/sec per suite:

SuiteOllama tok/sNInfer tok/s
AT1133 (137)412 (491)
AT2131 (135)449 (490)
AT3155 (159)694 (696)
AT4135 (135)541 (531)

A fairness note that cuts in NInfer's favor, not against it: the Ollama figure is a pure decode rate (eval_count / eval_duration, prefill excluded). The NInfer figure is output tokens divided by total wall time — prefill, network round-trip, everything included. NInfer's real decode-only rate is therefore higher than the number shown here; the ~4–5× gap above is, if anything, an underestimate of the engine's raw speed advantage. Isolated single-request samples against NInfer reached 460–620 tok/s.

Parallelism does not help much on either side at this hardware tier. Running 8 or 16 concurrent requests against NInfer barely increases aggregate throughput — a single request already saturates the RTX 5090 for a model this size. If your workload is “one interactive user,” this is not a limitation; if you were hoping to batch many requests per GPU, it is worth knowing before you plan around it.


Bonus: The VT Pre-Checks (VT1–VT9)

Beyond the AT-series, the harness runs nine smaller pre-checks (instruction discipline, JSON-schema fidelity, reproducibility, context isolation, tool-name alignment, Delphi-migration-risk keyword detection, tool-use-over-guessing, thinking on/off, and large-scale tool naming). Every VT that has a pass/fail verdict passes identically on both engines. Two results stand out beyond pass/fail:

  • VT9 Phase 1 (tool naming against 198 MCP tools): Ollama scores PARTIAL (exact 28 / semantic 85 / different 81 / missing 1), NInfer scores PASS (exact 51 / semantic 122 / different 25 / missing 0) — a real quality gap on this specific task, not just a speed difference.
  • Wall-clock on VT1–VT7: NInfer finishes in roughly a tenth to a hundredth of Ollama's time (e.g. VT1: 13.5 s → 0.1 s, VT7: 12.8 s → 0.2 s) — consistent with the AT-series speedups, and a reminder that for short, low-token-count prompts the fixed overhead Ollama carries matters more than raw decode speed.

Follow-Up: Qwen3.8-27B — Thinking Disabled, Plus INT8/252k Context

The two threads left open above — the dense Qwen3.8-27B numbers and the INT8/256k-context option — are now measured, not estimated. The earlier *_ninfer_qwen3.8-27b-nvfp4.json files (run with thinking) are superseded and were not used below: at temperature 0, thinking made Qwen3.8 run to the 8192-token output limit without ever answering in 14 of 30 AT3 tasks. Both new NInfer runs disable thinking (enable_thinking=false); without it, AT3 answers average 238 output tokens instead.

Setup. Ollama side: qwen3.8:27b-q4_K_M, existing results from 2026-08-29/30, not re-measured for this post. That run already used a large context (prompts up to 188k tokens, no silent truncation observed) and /no_think in the prompt, so it did comparatively little thinking either. Whether Ollama offloaded to CPU in that run was not checked — unlike the qwen3.6:35b-a3b correction earlier in this post, no claim is made either way here.

NInfer side: v0.9.0, model qwen3_8_27b_nvfp4 (.ninfer, NVFP4), both runs without thinking, temperature 0:

  • Run “bf16”: KV-cache in bf16, 150k context (default startup script).
  • Run “INT8”: --kv-dtype int8, --max-context 252928, truncation threshold 212992 to match the Ollama run.

Results (Ollama / NInfer bf16 / NInfer INT8; time = full suite):

Suite Ollama NInfer bf16 NInfer INT8 Unit Ollama time bf16 time INT8 time
AT163.665.7 (28/30)62.8 (30/30)Hit %4,309 s467 s449 s
AT1-DE49.738.7 (28/30)36.2 (30/30)Hit %1,616 s348 s320 s
AT279.8 (113 scored)72.8 (112 scored)71.7 (120 scored)Hit %18,116 s1,865 s2,026 s
AT385.380.982.3Structure+content %3,335 s53 s50 s
AT40.8020.8260.820Score 0–1467 s63 s56 s
AT51.0001.0000.996Score 0–1207 s46 s37 s
AT60.8060.8170.817Score 0–1279 s41 s33 s

AT1–AT4 combined: Ollama 26,227 s vs. NInfer ≈2,450 s — roughly 10× faster, on top of an already-fast model class.

On AT1 and AT2, the “all tasks” row above hides three different denominators, and that's worth spelling out rather than glossing over: AT1 and AT2 both have 120 (AT2) or 30 (AT1) nominal tasks, but each engine/run scored a different subset for a different reason. NInfer INT8 answered everything (120/120, 30/30). NInfer bf16 rejected 8 of AT2's tasks outright with context_length_exceeded — the two oversized files from earlier in this post — leaving 112/120. Ollama scored only 113/120 on AT2, but for a different reason: seven specific calls returned empty responses with prompt_tokens=0 and no error text — not context-size-related, since none of those are oversized files, and the exact cause (likely a timeout or abort during the 2026-08-30 run) can no longer be determined from the stored results. Because the three “all tasks” numbers aren't comparing the same task set, the paired comparison restricted to the 98 tasks all three completed is the cleaner AT2 figure: Ollama 81.8, NInfer bf16 73.4, NInfer INT8 71.7. The equivalent restriction for AT1 (26 shared tasks) gives 65.9 / 66.1 / 64.8.

VT pre-checks: VT1–VT8 pass/fail identically to Ollama on both NInfer runs, 3–100× faster (e.g. VT7: 48.4 s → 0.5 s). VT9 Phase 1 (exact tool-name matches out of 198 MCP tools): Ollama 65, NInfer bf16 76, NInfer INT8 77. VT9v2 has no built-in quality scoring — it's a timing check only (411 s vs. 15 s / 12 s) and should not be read as a quality result.

Token rates (mean per suite, AT1/AT2/AT3/AT4): Ollama 33/29/10/68, NInfer bf16 48/72/128/110, NInfer INT8 82/89/155/121 tok/s. The same caveat as the Qwen3.6 numbers above applies, more sharply here: Ollama's figure is decode-only; NInfer's is output tokens over total wall time including prefill — and with thinking disabled, answers are short, so prefill dominates NInfer's wall time more than in the Qwen3.6 runs. Use the suite wall-clock times above for the speed comparison, not these per-token rates.

Three takeaways:

  1. Speed: NInfer is roughly 9–10× faster than Ollama on Qwen3.8 at the same task volume.
  2. Quality is mixed, unlike Qwen3.6. AT1, AT3–AT6, and the VT suite are tied within noise. AT2 is the exception — NInfer trails Ollama by about 8 points (paired comparison), and AT1-DE by about 13 points, NInfer's weakest result in this whole post. This is the opposite pattern from the Qwen3.6 comparison earlier, where NInfer matched or beat Ollama on every metric. Possible causes — NVFP4 vs. Q4_K_M quantization, a chat-template mismatch, or something specific to German-language prompts — have not been investigated; treat this as an open question, not a diagnosed regression.
  3. INT8 KV-cache at 252k context costs essentially nothing and buys real headroom. Quality moves by at most ±1.5 points against bf16 (sometimes better, sometimes worse), speed is even or slightly faster, and — the actual point of testing it — no task is rejected anymore. The bf16/150k run hit context_length_exceeded on 12 tasks (AT1: 2, AT1-DE: 2, AT2: 8); the INT8/252k run answered all of them, including prompts up to 188,232 tokens. The model vendor's own README says this model “should stay at 150k” — the INT8 KV-cache evidently buys the full advertised context back without a quality cost worth mentioning.

For the headline comparison in this section, INT8 vs. Ollama is the one to read; bf16 is included as a secondary data point specifically to isolate what INT8 changes (quality: negligible difference; context ceiling: the whole story).


What This Means in Practice

The quality delta for Qwen3.6 should not be over-read: it is the same model, converted to a different runtime format and quantization scheme (groupwise-int vs. Q4_K_M), so some of the AT1/AT3 gain plausibly comes from quantization, not the engine per se — this post does not have a same-quantization, engine-only isolation to separate those two variables, and that is a gap worth closing before drawing firm conclusions about NInfer's format specifically. What is unambiguous is the throughput number: 4–10× faster wall-clock on identical benchmark suites, on the same GPU, across two different model families, with no quality regression on most metrics measured. For a pipeline currently paying Ollama's decode rate, that is a substantial, reproducible win, and the integration cost is low — OllamaMCP 1.6.0 already routes any model name prefixed ninfer/ to NInfer's OpenAI-compatible endpoint, so existing callers (email triage, chat) switch by changing a model name, not by rewriting integration code. A new NInferMCP module starts, stops, and checks the status of the NInfer server headlessly, without a console or web UI dependency.

The truncation finding deserves to travel independently of the speed story: if you have been benchmarking or running any local model against source files near or above your configured num_ctx, check whether your engine is silently truncating rather than telling you. Ollama's behavior here — accepting the oversized prompt and returning prompt_tokens capped at exactly num_ctx with no error — is easy to miss and directly invalidates any benchmark result computed on the truncated remainder. It should be looked for in any pipeline handling large Delphi units, not just this one.

What this post does not claim: that NInfer's .ninfer quantization format is categorically better than GGUF Q4_K_M — that requires an engine-only comparison at matched quantization, which does not exist yet. The dense Qwen3.8-27B result above is measured, not preliminary, but it opens its own open question rather than closing one: why NInfer trails Ollama specifically on AT2 and AT1-DE for this model, when it matched or beat Ollama on every metric for qwen3.6:35b-a3b. That is a genuine gap in this post, not a hedge — worth a targeted follow-up rather than a guess here.


Note on the Use of AI: This article was created with the "assistance" of generative AI. The content, technical statements, and conclusions have been reviewed and revised by the author. The author bears full responsibility for the publication.