There is in fact an X-Prize for AI, and it is general enough that it can apply to safety/alignment, though it's not purely aimed at this. http://ai.xprize.org
www.OpenEth.org is a 'let's build stuff and iterate whilst flailing around slightly as concepts develop' engineering-focussed project for Narrow AI / Smart Contract / Algorithm alignment and proofing, that is competing for this prize.
AISC has been an incredible wellspring of support for my AI Safety projects over several cohorts. They provide a wonderful opportunity for talented individuals who otherwise would not have funding to work on worthy projects, and AISC offers incredible structure and support to make these projects as impactful as possible. I have made so many connections that have been instrumental to my research.
I cannot recommend AISC highly enough. Their track record of nurturing important work in AI Safety deserves strong, continued support from the community.