I've observed the same while fine tuning the latest OpenAI chat model, GPT-3.5. It's very bad. The Da Vinci model has no protections in place whatsoever.
I plan to work on an open-source solution for this issue over the next few weeks. If I make any improvements to the alignment of my models, I'll update here or post it on the forum!