Hi, I'm a senior high school student and this is my first post here. I’m excited (and a bit nervous) to share a concept I’ve been working on and to hear your thoughts. I hope this post can spark some discussion or even lead to suggestions for improvement or collaboration.

Background & Motivation

This idea is entirely my own — I didn’t refer to any existing work when it first came to mind. I understand that it’s rare for high schoolers to propose new architectural concepts, so I expect skepticism, but I’ve spent a long time developing this and would really appreciate thoughtful feedback.

Core Idea

I call the framework VATS (Vector-Aligned Token Slicer). It’s a conceptual system that tries to categorize tokens and organize them in a layered three-dimensional space — not in a literal vector space sense, but more like a structured, logical domain map.

Imagine a cube with infinite side length. I “slice” this conceptual space into domain-specific layers (e.g., medical, literary, scientific). Each slice contains tokens relevant to that domain, and overlapping regions contain tokens shared across multiple domains.

Given a prompt, VATS identifies relevant slices using semantic similarity. It then selects tokens not only from the most related slice but also from overlapping slices, helping reduce ambiguity and preserve both domain specificity and semantic consistency. For example, if a word has multiple meanings across fields, VATS helps choose the interpretation aligned with the prompt’s intent.

Training & Generation

VATS doesn’t aim to replace traditional embedding-based models. Instead, it adds a spatial filtering layer.

During training:

Tokens are embedded using a base model like BERT.

These embeddings are clustered into slices using domain labels, semantic distances, or unsupervised methods.

A spatial grid is constructed where these slices reside.

During generation:

The model traverses slices based on prompt relevance.

Tokens from shared areas are prioritized when ambiguity arises.

The system refines itself through iterative alignment and slice adjustment.

Current Progress

I’ve implemented a basic diffusion-based language model from scratch. It currently supports:

Training on text datasets

Outputting training loss, epoch info, etc.

However, I’m still struggling with how to properly save and restore training results, especially in a way that supports multi-st...

Posts

Wikitag Contributions

Comments