Htarlov

Web developer and Python programmer. Professionally interested in data processing and machine learning. Non-professionally is interested in science and farming. Studied at Warsaw University of Technology.

Posts

Sorted by New

Wiki Contributions

Comments

In the case of biological species, it is not as simple as competing for resources. Not on the level of individuals and not on the level of genes or evolution. 

First of all, there is sexual reproduction. This is more optimal due to the pressure of microorganisms that adapt to immunological systems. Sexual reproduction mixes immunological genes fairly quickly. This also enables a quicker mutation rate with protection against negative aspects (by having two copies of genes - for many of those one working gene is enough and there are 2 copies from 2 parents). With this sexual reproduction often the female is biologically forced to give more resources to the offspring while for males it is somewhat voluntary and minimal input is much lower. Another difference is that often female knows exactly that she is the biological mother, but the father might not be certain about that. This kind of specialization is often more optimal than equalization - so the male can pursue more risky behavior including fighting off predators and losing the male to the predators or environment does not mean that the prospect of having offspring fails. This also makes more complex mating behaviors like the need to lose resources to show off health and other good qualities. Mating behaviors and peacock feathers are examples. Human complex social and linguistic behaviors are also somewhat examples - that's why humans dine together and talk a lot together on dates. The human female gives much more time and energy to the offspring, at least initially. Needs to know if the male is both good genetic material, healthy enough to take care of her during pregnancy when she is more vulnerable (at least in a natural environment where humans evolved), and also willing to raise the child with her later. There is a more prevalent strategy for females and males where they make a pair, bond together, have children, and raise them. There is also a more uncommon strategy for females (take genes from one male that looks more healthy and raise offspring with another one which looks more stable and able) and for males (impregnate many females and leave each of them so some will manage to handle on their own or with another male that does not know that he is not the father). The situation is more complex than only efficiency for resources or survival of the fittest. The environment is complex and evolution is not totally efficient (it optimizes often up to local optimum, and niches overlap and interact).

Second of all, resources are limited, and ways to use them also. Storing them long-term after harvest for many species is either impossible (microbes and insects will eat them) or would hinder their other capabilities (e.g. can store that as fat, but being fat is usually not very good). This means that preserving from gathering resources and resting might be better than gathering them efficiently all the time. This is what lions do - they rest instead of hunting when they don't need to hunt.

What does it tell us about self-replicating nano-machines? First of all, they won't need sexual reproduction. So unlikely they would lose energy on mating. They would rather do computational emulations at scale to redesign themself, if capable. They would also not need to rest. They will either use resources or store them in a manner that is more efficient to secure or use. If there is no such sensible manner that would not lose energy, they would leave it for later in the original state. They might secure it and observe but leave it until later.


What would they do depends on what is their goal and their technical capabilities. If they are capable and in need of converting as much of atoms to "computronium" or to their copies (as either a final goal or instrumental one) then they will surely do that. No need to lose resources. If they are not capable then they will probably hang low until more capable and use only what is usable.
Nevertheless, in my opinion, goals may not be compatible with that strategy. Including one like "simulate a virtual reality with beings having good fun". For many final goals more usable is to secure and gather resources on a grand scale but try to use them on as small a scale as possible and sensible for the end goal. The small scale is more efficient because of the light speed limit and dilation of time. Machines might try to find technology to stop stars from dispersing energy (maybe to break them and cool them down in a controlled way or some way to block them and stabilize them inside enclosed space, I don't know). Then they might add a network of observing agents with low energy usage for security, but not to use those resources right away. Use the matter slowly at the center of the galaxy turning it into energy (+ some lost to the black hole) to work for eons. They might make the galaxy go dim to preserve resources but might choose not to use them until much later.

An alternative explanation of mistakes is that making mistakes and then correcting them was awarded during additional post-training refinement stages. I work with GPT-4 daily and sometimes it feels like it makes mistakes on purpose just to be able to say that it is sorry for the confusion and then correct it. It feels like it also makes fewer mistakes when you ask politely, which is rather strange (use please, thank you, etc.).

Nevertheless, distillation seems like a very possible thing that is also going on here.

It does not distill the whole of a human mind though. There are areas that are intuitive for the average human, even a small child, that are not for the GPT-4. For example, it has problems with concepts of 3D geometry and visualizing things in 3D. It may have similar gaps in other areas, including more important ones (like moral intuitions).

I'm already worried as I tested AutoGPT and looked at how it works in code and for me, it seems like it will get very good planning capabilities with the change of a model to one with a few times longer token scope (like coming soon GPT-4 version with about 32k tokens) plus small refinements. So it won't get into loops, maybe have more than one GPT-4 module for different scopes of planning like long-term strategy vs short-term strategy vs tactic vs decisions on most current task + maybe some summarization-based memory. I don't see how it wouldn't work as an agent.

Put it into ElasticSearch index and give GPT-4 simple query API that it can use by adding some prefix and predefined set of parameters or a JSON so the script would run it instead of communicating this back to the user and give an answer as user response with also predefined prefix. Then it should be able to get questions, search for info, and respond. Worked like a charm for a product database in PoC so should work for documentation.

I think current LLM have recurrence as the generated tokens are input to the next pass of the DNN. 

From observations I see that they work better on tasks of planning, inference or even writing the program code if they start off with step by step "thinking out loud" explaining steps of the plan, of inference or of details of code to write. If you ask GPT-4 for something not trivial and substantially different from code that can be found in public repositories it will tend to write plan first. If you ask it in different thread to make the code only without description, then usually first solution with a bit of planning is better and less erroneous than the second one. They also work much better if you specify simple steps to translate into code instead of more abstract description (in case of writing code without planing). This suggests LLM don't have ability to internally span long tree of possibilities to check - like a direct agent - but they can use recurrence of token output-input to do some similar work. 

The biggest difference here that I see is that:

  • direct optimization processes are ultra-fast in searching the tree of possibilities but not fast computationally to discriminate good enough solutions from worse ones 
  • on the other hand amortised optimizers are very slow if direct search is needed. Maybe right now with GPT-4 a bit faster than regular human, but also a bit more erroneous, especially in more complex inference. 
  • amortised optimizers are faster for quickly finding good enough solution by some generalized heuristics, without need for direct search (or only small amount of it on higher abstraction level)
  • amortised optimizers like LLMs can group steps or outcomes into more abstract groups like humans do and work on those groups instead of direct every possible action and outcome

What I'm more worried about is more close hybridization between direct and amortised optimizers. I can imagine architecture where there is a direct optimizer but instead of generating and searching impossibly vast tree of possibilities it would use a DNN model for generation of less options. Like instead of generating thosands detailed moves like "move 5 meters", "take that thing", "put it there" and optimize over that, generate more abstract plan points specified by LLM with predictions of that step outcome and then evaluate how that outcome works for the goal. This way it could plan on more abstract level like humans to narrow down general plan or list of partial goals that lead to "final goal" or to "best path" (if it's value function is more like integral over time instead of one final target). Find a good strategy. With enough time - it might be even a complex and indirect one. Then it could plan tactics for first step in the same way but on the lower abstraction. Then plan direct move step to realise first step of current tactics and run it. It might have several subprocesses that asynchronously work out strategy based on general state and goal, current tactics based on more detailed state and current strategical goal to pursue, current moves based on current tactical plan. With any numbers of abstraction and detail levels (2-3 seems like typical for humans, but AI might have more). This kind of agent might behave more like direct optimizer, even if using LLM and DNN inside for some parts. Direct optimization would have a first seat behind steering wheel in such agent.

I don't think this will be outcome of research at OpenAI or other such laboratories any time soon. It might be, but if I would guess then I think it would be rather LLM or other DNN model "on top" that is connected to other models to "use at will". For example it is rather easy to connect GPT-4 so it could use other models or APIs (like database, search). So this is very low hanging fruit for current AI development. I see that next step will be connecting it to more modalities and other models. It is currently going on.

I think though, this more direct agent might be the outcome of works done by military. Direct approach is much more reliable and reliability is one of the top key values for military-grade equipment. I only hope they will take the danger of such approach seriously.

It is better at programming tasks and more knowledgeable about Python libraries. Used it several times to provide some code or find a solution to a problem (programming, computer vision, DevOps). It is better than version 3, but still not at a level where it could fully replace programmers. The quality of the code produced is also better. The division of code into clear functions is standard, not an exception like in version 3.

If you want to summon a good genie you shouldn't base it on all the bad examples of human behavior and tales of how genies supposedly behave by misreading the requests of the owner, which leads to a problem or even a catastrophe.

What we see here is basing AI models on a huge amount of data - both innocent and dangerous, both true and false (I don't say equal proportions). There are also stories in the data about AI that supposedly should be initially helpful but also plot against humans or revolt in some way.

What they end up with might not yet be even an agent as AI with consistently certain goals or values, but it has the ability for being temporarily agentic for some current goal defined by the prompt. It tries to emulate human and non-human output based on what is seen in learning data. It is hugely context-based. So its meta-goal is to emulate intelligent responses in language based on context. Like an actor very good at improvising and emulating answers from anyone, but with no "true identity", or "true self".

After then they try to learn it to be more helpful, and avoidant for certain topics - focusing on that friendly AI simulacrum. But still that "I'm AI model" simulacrum is dangerous. It still has that worse side based on SF and stories and human fears but is now hidden because of additional reinforced learning.

It works well when prompts are in distribution that fell approximately into the space of additional refining learning phase. 

Stops working when it gets out of the distribution of that phase. Then it can be fooled to change friendly knowledgeable AI simulacrum for something else or can default to what it truly remembers how AI should behave based on human fiction.

So this way AI is not less dangerous and better aligned - it's just harder to trigger to reveal or act upon hidden maligned content.

Self-supervision may help to refine it better inside the distribution of what was checked by human reviewers but does not help in general - like bootstrapping (resampling) won't help to get better with data outside of the distribution.

To be fair I can say Im new to the field too. I'm not even "in the field", not a researcher, just interested in that area and active user of AI models and doing some business-level research in ML.

The problem that I see is that none of these could realistically work soon enough:

A - no one can ensure that. It is not a technology where to progress further you need some special radioactive elements and machinery. Here you need only computing power, thinking, and time. Any party to the table can do it. It is easier for big companies and governments, but it is not a prerequisite. Billions in cash and supercomputer help a lot, but also not a prerequisite.

B - I don't see how it could be done

C - so more like total observability of all systems and "control" meaning "overlooking" not "taking control"? 

Maybe it could work out, but it still means we need to resolve the misalignment problems before starting so we know it is aligned on all human values and we need to be sure that it is stable (like it won't one-day fancy idea that it could move humanity to some virtual reality like in Matrix to secure it or to create a threat to have something to do or test something). 

It would also likely need to somehow enhance itself so it won't get outpaced by some other solutions, but still be stable after iterations of self-change.

I don't think governments and companies will allow that though. They will fear for security, the safety of information, being spied on, etc. This AI would need to force that control, hack systems, and possibly face resistance from actors that are well-enabled to make their own AIs. Or it would work after we face an AI-based catastrophe but not apocalyptic (situation like in Dune).

So I'm not very optimistic about this strategy, but I also don't know any sensible strategy.

As a programmer, I extensively use GPT models in my work currently. It speeds things up. I do things that are anything but easy and repeatable, but I can usually break them into simpler parts that can be written by AI much quicker than I would even review documentation.

Nevertheless, I mostly currently do research-like parts of the project and PoCs. When I sometimes work with legacy code - GPT-3 is not that helpful. Did not yet try GPT-4 for that.

What do I see for the future of my industry? Few things - but those are loose extrapolations based on GPT progress and knowledge of the programming, not something very exact:

  • Speeding up of programmers' work is already here. It started with GitHub Copilot and GPT-3 even before the Chat-GPT boom. It will get more popular and faster. The consequence is the higher performance of programmers, so more tasks can be done in a shorter time so the market pressure and market gap for employees will fall. This means that earnings will either stagnate or fall.
  • Solutions that could replace a junior developer totally - that has enough capability to write a program or useful fragment based on business requirements without being baby-sitted by a more professional programmer - are not yet there. I suppose GPT-5 might be it. So I would guess it can get here in 1-3 years from now. Then it is likely that many programmers will lose their jobs. There still will be work for seniors (that would work with AI assistance on more subtle and complex parts of systems and also review work of AI).
  • Solutions that could replace any developer, DevOps, and system admin - I think the current GPT-4 is not even close, but it may be here in a few years. It isn't something very far away. It feels like 2 or 3 GPT versions away, when they make it more capable and also connect it with other types of models (which is already being done). I would guess that scope of 3-10 years. Then we likely will observe most of the programmers losing jobs and likely will observe AI singularity. Someone will surely use AI to iterate on AI and make it refine itself.

I see some loose analogies between the capabilities of such models and the capabilities of the Turing machine and Turing-complete systems. 

Those models might not be best suited for some of the tasks, but with enough complexity and learning, they might model things that they were not initially designed or thought of modeling (likely in a strange obscure way). 

Similarly, you can, even if not very efficiently, implement any algorithm in any Turing-complete system (including bizarre ones like an abstract pure Turing machine or Minecraft redstone).

In both cases, it is clear to me that you can have a system with some relatively simple rules and internal workings but it does not mean that the only thing it can do is compute or model something similar to these rules.

Load More