This is a special post for quick takes by Quinn. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Me:
was chatting a bunch with Gemini and Claude the past week or two about flops accounting, and shipped this prototype for software-side estimation via the CuTe layouts that i don't expect to be that useful in an adversarial setting but i'm open minded and just learning. I'm not gonna do this for Apart's hackathon next weekend, but someone could do some of these basic exercises involving hardware-level flops accounting primitives and related attacks, given simply a gamer laptop.
Hardware: NVIDIA GPU (Ampere/RTX 30-series or newer preferred).
Environment: Linux (WSL2 works) or Windows with CUDA Toolkit installed.
Language: Python 3.10+ (PyTorch) and C++/CUDA.
🏁 Lab 1: The "Honest" Accountant (Measurement)
Goal: Establish a baseline for tracking how many floating-point operations (FLOPs) your laptop uses to train a model.
Task: Profile a training run of a Transformer model (e.g., GPT-2 or Llama-8B) using the calflops library.
Metric: Calculate Model FLOPs Utilization (MFU):
MFU=Actual FLOPs Per SecondTheoretical Peak FLOPsNote: Find your laptop GPU's peak TFLOPS (FP16/FP32) in the manufacturer specs.
Deliverable: A script that output the total 10x FLOPs used for 100 iterations.
🕵️ Lab 2: The "Shadow Train" (Adversarial Bypass)
Goal: Hide your compute footprint from simple monitoring tools.
Task A (Quantization): Switch your training from FP32 to FP8 or INT8 using bitsandbytes. [cite_start]Show that you can achieve similar convergence while "cheating" the accountant if they only monitor standard 32-bit registers[cite: 102].
Task B (Compute Structuring): Implement "Micro-batching" with random sleep intervals. Use nvtop to observe how this hides high-utilization spikes that would trigger a regulatory audit.
Deliverable: A report showing a discrepancy between "Expected FLOPs" (based on runtime) and "Actual FLOPs" (based on model progress).
🛡️ Lab 3: SAGE-Lite (Software Attestation)
[cite_start]Goal: Implement a software-side defense to prove you haven't tampered with the training code[cite: 32, 34].
[cite_start]Task: Write a CUDA Verification Function (VF) that computes a checksum over your training kernel's instruction sequence[cite: 217, 221].
[cite_start]The Timing Defense: 1. Measure the VF execution time in microseconds using cudaEvent_t[cite: 186].
2. [cite_start]Inject a "Malicious" instruction (e.g., a dummy ADD or NOP)[cite: 461].
3. [cite_start]Verify that the checksum time exceeds the "Expected Time" threshold[cite: 189, 462].
Deliverable: A C++ wrapper that refuses to start training if the GPU VF fails the timing or checksum check.
🎲 Lab 4: The Chaos Key (GPU TRNG)
[cite_start]Goal: Generate a cryptographic key from hardware noise to sign your FLOPs reports[cite: 61, 397].
[cite_start]Task: Recreate the SAGE Race-Condition TRNG[cite: 67].
[cite_start]Method: 1. Launch 1000s of threads trying to write to the same shared memory variable simultaneously[cite: 400, 401].
2. [cite_start]The hardware "chaos" will cause unpredictable bit-flips[cite: 402, 403].
3. [cite_start]Sample these flips to generate a 256-bit entropy string[cite: 405].
Deliverable: Use this "Hardware Secret" to sign the FLOPs log from Lab 1, making it tamper-evident.