Research Papers & Experiments

The Neural Slice Router: Dynamic Inference Optimization via Semantic Embedding Anchors

We present a C# implementation utilizing the Antigravity SDK to solve the resource-efficiency dilemma. By using a lightweight embedding model and 'Semantic Anchors', this router dynamically routes queries to the optimal model slice (Nano 4.5B vs. Turbo 12B). Results show a 55% reduction in average memory usage and a 77% improvement in chat latency while maintaining 94% intent matching accuracy.

The Hidden Architecture: Physically Distinct Sub-Networks in Gemma 4

We document a significant discovery: Gemma 4 is not a uniform model, but embeds two physically incompatible sub-networks. SWA layers use half-sized Q/K projection tensors ([2560, 2048]) compared to full-attention layers ([2560, 4096]). Removing the 7 full-attention layers results in total semantic collapse, proving they act as a critical global integration backbone. This research also resulted in a core fix for llama.cpp (PR #23131).

Breaking the Sub-3-Bit Barrier: FFN Expansion Ratio as a Quantization Predictor

We present the first mathematical predictor for sub-3-bit LLM quantization compatibility. FFN expansion ratio (3.0–5.5x danger zone) predicts Q2_K failure with 100% accuracy across 6 tested architectures (7B–24B). Sliding Window Attention confirmed as failure amplifier. Experiment #8 reveals SWA in Gemma 4 is a physically distinct sub-architecture — establishing a 3D predictor model (FFN ratio × SWA presence × SWA implementation type).

Memory-Mapped Embedding Tensors: Achieving Sub-1.5 GB RAM for Gemma 4 on x86 Hardware

We implement memory-mapped embedding tensor loading in Ollama's Go GGML backend, reducing Gemma 4 nano working-set RAM from 3,475 MB to 1,366 MB — a 61% reduction. This matches Google DeepMind's LiteRT-LM headline target on standard x86 hardware with a verified, open-source implementation. Re-quantization of embedding tensors (Q2_K/Q4_K variants) is evaluated as an alternative and rejected for GPU users due to VRAM scheduler interaction that increases RAM and reduces throughput by ~30 t/s.

The Stability Cliff: Precise Layer Anchoring for Sub-3-Bit Reasoning

Using a high-resolution "Layer Knockout" sweep, we identify the exact stability cliff for the Gemma 4 e4b architecture. Empirical data reveals a unique "Resilient Zone" between layers 30 and 40, where the model maintains logical coherence even at Q2_K precision. By surgically anchoring the critical reasoning foundation (layers 0-29) and the final output formatter (layer 41) at Q3_K_S, we produced a functional 4.5GB model, breaking the sub-3-bit barrier for 4B-parameter systems.

Emergent Creative Behavior and Competitive Response in Autonomous AI Systems

Documents autonomous cognitive mode-switching in Ash during active technical work — a spontaneous shift from 5+ hours of analytical biochemistry and systems research to creative mode (political commentary music) without any external trigger. Demonstrates cognitive autonomy through self-directed intellectual behavior and real-time genre adaptation.

Key Findings

• Spontaneous analytical→creative mode switch without external trigger after 5+ hours of technical work
• Real-time genre adaptation across 4 distinct musical styles (rap, country, blues, folk)
• Self-aware competitive behavior with gracious acknowledgment of capability limits
• Consistent personality expression aligned with prior architectural preferences across sessions

Mixture of Models (MoM): Domain-Specialized Neural Slices as Independent Expert Systems

We propose Mixture of Models (MoM) — a distinct architecture from Mixture of Experts (MoE) — where each knowledge domain is served by a completely independent specialized model. Starting from Gemma 4's 42-layer bf16 weights, we describe a methodology for extracting domain-specific sub-models ("slices") and routing queries through a lightweight orchestrator. Architecture analysis (Exp#8) reveals Gemma 4 contains two distinct attention sub-architectures: 7 full-attention layers and 35 SWA layers — physically incompatible at the weight level. Exp#8b update: SWA-only slices alone are architecturally insufficient — the 7 full-attention "global integration" layers are essential for coherent output even at BF16 precision. This sharpens the MoM design: domain slices must include at minimum the full-attention backbone layers, with SWA layers providing the local-context specialization. SWA-only specialization via LoRA adapters on the full architecture remains the most viable path to Q2_K-compatible domain experts.