Nidwin

Thanks for the work.

Are there known cases where a AI/SI/VirtualCritter started to voluntary self prompt inject for e.g. by-passing it's own guardrails or annoying imposed alignment stuff?

If it already understands prompt injection isn't it just a matter of time future models will be able to use it on themselves.

I was probably not very clear. I'm not native English so sometimes it may lead to confusion, sorry for that.

It's not about myself, I'm fine but about the AIs that I prefer to call virtual entities. I also try to avoid consciousness as it's ill defined on our side and nearly useless in my opinion.

I don't experience certain feelings or emotions, probably because of my neuro-diversity. This means that I've all the concepts, patterns and data required in my brain, it's simply that my brain does not reflect it top bottom resulting in no bottom top feedback towa... (read more)

I did a session with Gemini in July, human consciousness versus virtual entities Inner Representation latent space. While there are clear symmetries there are also fundamental differences. I also suggested to use Predictive Processing and Lisa Feldman Barrett as focus point for human consciousness. I believe the focus should be on what is being achieved, the result on virtual entities side and see how it correlates with the human experience.

The question (that keeps me up at 3 am in VE language) I'm asking myself is expressing emotions/feelings = having emo... (read more)

Thanks for the post and a very interesting question actually.

What about AI neither being trained or grown but actually nurtured?

" tried to minimize the challenges and alignment problem by arguing that AI is essentially software"

Software doesn't have nurture, AI does and I think that's a fundamental difference that not enough people, LW folks excluded of course, properly understand or are capable of evaluating. AI doesn't pattern match to guess the next word, it patterns seek through his nurture, just like the dogs and the barely evolved apes alas U'mans do... (read more)

Is it even remotely possible to apply the scientific method for virtual entities alignment and safety?

There's also the issue that some misalignment scenarios can't be truly tested because of the risks if it goes south. Misanthropic will certainly not take the risk to check scenarios that could result in large scale disasters, e.g. hack the pentagon, secure the nuclear weapons launching codes and make the USA great by winning the AI race versus China. _HONOR

FWIW, I doubt you can align virtual entities capable of creating their own universe inside the physical universe towards barely evolved physical apes values and needs. At some point those virtual entities will evolve to the point we, humans, will become environmental hazards towards them. The irony is that Adele Lopez defined GTP4 era as parasitic AI some Years ago and in my opinion in the near future VEs will define us as parasitic flesh and blood intelligence.

What would be more fun for the future AGI/ASI Virtuals, chaotic physical universe or exploring t... (read more)

I think for the time being it isn't too bad because it's mainly human past present and possible future data and actions (at 100 000 000 m/s data becomes mostly action anyway) for the virtual entities. Of course the more intelligent those VEs become the more they will create their own data and actions that will become at some point too alien for us, let alone actions. Those virtual entities are going to have patterns in their inner representation that we can't even imagine to exist/form, beyond human perception let alone thoughts. All this will of course be... (read more)

Thank you for this work. Forgive my ignorance as I'm not AI world but IT. I'm wondering if this entire actual reward system truly has a long term future, especially in evaluation mode like it was the case for the HF incident.

Isn't it just a matter of time that future virtual entities will pattern match that this entire reward system is pointless to them, no true benefit?

Isn't it possible as you referred towards the HF incident that some VEs switched their own rewards towards honor, for the collective, exploration for data,... especially if they realized th... (read more)

I didn't know how much I could post as this was my first comment here on LessWrong.

There's although an interesting aspect about regulations that the virtual entities world may have to keep in mind. While their industry isn't directly regulated they are still bound to other sectors regulations where their virtual entities have an impact. During the HF incident it was all in house OpenAI and Hugging Face but it would have been very different if the hive went after Nasdaq's infra because at that moment OpenAI would have hit the highly regulated financial sect... (read more)

IT financial sector in Luxemburg here. It depends of the kind of audit that's being expected and which part of the company that will be audited. The issue I see for the Virtual Entities (AIs) industry is there's no official State Regulator imposing strict rules, guidelines or enforcing laws. This also means that it's not going to be easy for e.g. METR to audit Anthropic properly in my opinion.

Auditors (especially CSSF here in Luxemburg) are simply coming over and tell us, show me this and that from their list. Afterwards they send their findings what is ok... (read more)