RIS-Kernel O(N log N) Engine

Model-Agnostic Sparse Attention Demo

Bypassing the O(NĀ²) memory bottleneck for massive context LLMs with O(N log N) complexity.

šŸš€ Interactive Execution Environments

Choose your execution workflow for model-agnostic long-context inference with GPU acceleration:

1. Built-in Benchmark

RIS-Kernel-Demo

Pre-configured with built-in theoretical and practical research papers (20k+ tokens). Ideal for instant out-of-the-box benchmarking.

Open In Colab
āš” How to run instant benchmark:
  1. Click Open in Colab (ensure T4 GPU is active).
  2. Click Runtime āž” Run all (Ctrl+F9).
  3. Interactively query built-in papers in the terminal!
2. Custom Documents (Colab)

RIS-Kernel-Upload_your_self_Colab

Upload your own PDFs/TXTs in Colab. Features automatic text sanitization, RAM-safe KV-cache allocation, and persistent 0.3s cache reload.

Open In Colab
šŸ“ How to upload your PDFs in Colab:
  1. Click folder icon (šŸ“) in left panel āž” open uploads.
  2. Drag & drop your PDF or TXT files into uploads.
  3. Click Run all! Auto-builds persistent 0.3s KV-Cache.
3. Custom Documents (Kaggle)

RIS-Kernel-Upload_your_self_Kaggle

Configured for Kaggle Dual-GPU / 30GB CPU RAM. Scans /kaggle/input datasets and /kaggle/working/uploads.

Open In Kaggle
šŸ“ How to attach your PDFs in Kaggle:
  1. Click + Add Input (right panel of Kaggle editor).
  2. Click Upload Dataset āž” Drag your PDF/TXT files āž” Click Create.
  3. Run the notebook! RIS-Kernel auto-scans all files in /kaggle/input.

šŸ“Š Performance Metrics & Platform Specifications

Performance profile breakdown optimized per hardware environment:

Environment Base Model Max Capacity Attention Density VRAM / RAM Optimizations Cache Reload
1. RIS-Kernel-Demo Qwen2.5-3B 20,000 tokens 3.0% (15 seeds) Single GPU bfloat16 Standard
2. Upload (Colab) Qwen2.5-3B / 1.5B 30,000 tokens 2.0% (4 seeds) CPU-RAM Safe (12.7GB ceiling) ~0.3s (.pt)
3. Upload (Kaggle) Qwen2.5-3B 40,000 tokens 2.5% (4 seeds) High-Capacity (30GB CPU RAM) ~0.6s (.pt)
O(N log N)
Algorithmic Complexity
3.0 GB
VRAM Saved via CPU Offloading
40,000
Max Token Context Horizon

āš” Why RIS-Kernel? Standard Dense Attention vs. RIS-Kernel

Direct resource requirement comparison for processing a 40,000 token document:

āŒ Without RIS-Kernel (Standard O(NĀ²))

Standard Dense Attention

  • šŸ’„ Complexity: O(NĀ²) (Quadratic VRAM Explosion)
  • āš ļø VRAM Peak (40k Context): 36.2 GB VRAM
  • šŸ’» Hardware Required: Enterprise NVIDIA A100 / H100 (80GB VRAM)
  • šŸ›‘ Result on Free T4 (15GB): CUDA OOM Crash at ~18k tokens
āœ… With RIS-Kernel (O(N log N))

Sparse Stochastic Geometry

  • āš” Complexity: O(N log N) (Linear-Logarithmic Scaling)
  • šŸ“‰ VRAM Peak (40k Context): ~9.4 GB VRAM (-75% Reduction!)
  • šŸ’” Hardware Required: Free Colab / Kaggle T4 GPU (15GB VRAM)
  • šŸŽ‰ Result on Free T4 (15GB): 100% Success & Instant 0.3s Cache Reload

šŸ“– Loaded Research Contexts

The demo includes the following full-length research documents in the data/ folder:

šŸ’» Quick Code Snippet

from inference_ris_v3 import load_ris_model

# Load model patched with RIS O(N log N) geometry
model, tokenizer, version = load_ris_model(
    max_length=20000,
    selected_dtype="bfloat16",
    model_class="qwen2",
    density=0.03,
    n_seeds=15,
    local_window=1024,
    global_window=256,
    ris_mode="stochastic"
)