I see a lot of marketing about "max input tokens" being in the hundreds of thousands or millions of tokens, but I have a theory that this only works with simple prompts like "summarise this data, here is the data...". If you have a prompt without a strong directive, made...
In a few sentences, what i'm doing is writing a computer program that constructs a question (a prompt), which is sent to Anthropic's Claude Sonnet, then i'm processing its output into actions, and the computer program then runs those actions on itself. It would be good if you have thoughts...
I wonder if the technique can be used to get around guardrails on open weight models. If you can get the activation pattern for some nefarious concept, you could activate it alongside an innocent prompt's pattern - maybe getting the llm to take more negative actions.
What actions can the agents actually take? Seems like they're just integrated with x and other platforms to write posts?
How are memories handled? What about goals and predictions? Or how are they coordinating?
I think a trap that I can fall into with ai systems is that they often produce good looking output, but it's ultimately surface level and not that deep.
I think the answer here depends on how you define "its assistant". What kind of system do you have in mind? I think this is an interesting question from a cyber security perspective.
Here's the Replit CEO Amjad Masad confirming what i've seen (timestamp: 36:45). "After 32k tokens, reasoning and a lot of benchmarks tank"
Thanks for your comment, I took a look at your example, but i'd say that is addressing a different issue - constrained output tokens, not ingestion of input tokens. I also wanted to avoid scaffolding approaches since i'm zero shotting, I don't want to use a chained series of prompts or chunking, I want to submit a single prompt.
I'm looking for any techniques similar to including an index of the prompt sections (like in a book with a list of the chapters) for the prompt and some character strings that differentiate the prompt's sections. Here's an example of the top of my prompt:
... (read more)Time Now: 2025-05-09 21:46:07
=== System Context === Character Count: 5903
1.
I see a lot of marketing about "max input tokens" being in the hundreds of thousands or millions of tokens, but I have a theory that this only works with simple prompts like "summarise this data, here is the data...".
If you have a prompt without a strong directive, made up of sections with equal sizes, then you lose coherence very fast. Gemini 1.5 Pro lost coherence at 25k input tokens. Gemini 2.5 Pro loses coherence at 35k with my prompts.
Imagine you're trying to construct a "thought" for an llm, where actions are extracted from the response and ran. The prompt has sections like:
I believe that opensource advancements like R1 will drive wider adoption of ai systems.
I think that the pricing models will change soon. Everyone talks about cost per million tokens to contact a hosted service, but I think it'll switch to be cloud costs to provide infrastructure that can run models. Virtual machines running something like ollama.
This solves another huge problem, privacy and how prompt data is handled. If you're using an api to a hosted service you need to have a very good understanding of how your submitted prompt data is handled. This is key for organisations. I feel like the lack of understanding here is preventing widespread adoption, especially for communication... (read more)
If you're downvoting, could you say why? I'm new to this site but there are very few posts on ai like my own. There's a few on different ai tools, but what I have isn't a tool, the output generated by it is for itself.
It started out with the idea to make a system that can improve itself, without me having to prompt it to improve itself. The "question" is the prompt. And crafting the prompt is very difficult. So it's an experiment in constructing a question that a system can use to improve itself without having to explicitly say "improve your codebase" or "give yourself more actions or freedoms" or others like that. I want the llm to conjure up ideas.
So as the prompt increases in complexity, with different sections that could be categorised as kinds of human thought, will it get better? Can we find parallels between how the human minds works that can... (read more)
In a few sentences, what i'm doing is writing a computer program that constructs a question (a prompt), which is sent to Anthropic's Claude Sonnet, then i'm processing its output into actions, and the computer program then runs those actions on itself.
It would be good if you have thoughts on this, as it's philosophically an "ask the right question" task.
The biggest struggle is thinking of how to ask the question "this is you, improve you" without having to define an improvement, and without having how to describe the outcome you want. I want the system to improve itself without me having to direct it - for my curiosity.
I've decided to phrase the... (read 1457 more words →)
What sort of latency does it experience? How large are the prompts?