After reading some about how affordable housing is actually
implemented, it looks to me like rich people could exploit it to avoid
paying property and inheritance taxes, and generally get around the
means testing requirements.
Affordable housing is about renting or selling homes well below market
price, so if there were a large pool of affordability-restricted
properties there would be a lot of incentive for people to figure out
how to get around the spirit of the restrictions. I'm going to talk
about buying here, but renting has a lot of similarities.
A typical buyer restriction today (Somerville
is something like:
Buyers who meet these restrictions are entered into a lottery, and the
winner gets a 2-bedroom 2.5 bathroom 1,500 square-foot unit for $177k instead
Property taxes are also very low, ~$200/y instead of ~9k/y. 
These restrictions apply at purchase time: you have to have a
relatively low income and assets to qualify, but then there are no
further restrictions. This makes sense, because otherwise we would be
requiring poor people to stay poor, but it also allows a lot of
potential ways for rich people to 'legally cheat':
Intentionally keep a low income for several years. Three years
at $70k instead of $140k loses you $210k, but you'd save more than
that in property taxes alone long-term.
Arrange for deferred or speculative compensation. Stock that
vests in four years, stock options, start a startup.
Get training that gives you high earning potential, but don't
start your high paying job until after you have the house. This
training is effectively an asset, but it's very hard for the
affordable housing administrators to price it, so it's ignored.
Learn through self-study or apprenticeship to get around the
prohibition on students.
Postpone transfers to your children until after they have
qualified for affordable housing, since the income and assets of
relatives are not considered.
Buy land, take advantage of density bonuses, build a large 100%
affordable fancy building, and sell the units to your
just-out-of-school currently-low-earning children.
There are also longer-term issues around resale. You can sell to
anyone you want, as long as they meet the buyer restrictions and pay
no more than the legal maximum price. This means sellers are in a
position where they can effectively give a very large untaxed gift.
This could let parents transfer large amounts of wealth to their
children, untaxed.  You could also have problems with corruption, where
I buy your property for $200k, but then I sneak you an extra $100k so
you sell it to me instead of someone else.
Since these are implemented by deed restriction, they could be hard to
fix if they're getting exploited. It's also not necessarily obvious
whether or how much abuse there is, since the whole problem is that
based on the city's verification legitimately poor people and
artificially poor people look the same. (And what do we mean by
"artificially poor," and do we want to include children of bankers who
decide to become artists or low-paid academics?)
It's possible that the amount of hassle for the potential savings is
too low for it to be worth it for rich people to subvert. If 90% of
the units are used as intended and only 10% are tax shelters, I'd
consider it not great but probably still good. But I'm very nervous
about building a system that sets up so many opportunities for people
with good lawyers to get around the spirit of the rules.
 The property is assessed at a low value because the city sets
maximum resale prices. Since that's below the value of the city's residential
exemption you're taxed as if the property is worth just 10% of
it's assessed value. I calculate $8,830/year in property taxes for
the market rate unit (after the residential exemption) and just
$190/year for the affordable unit.
 Stow MA's Deed Restriction Program (faq)
is an example of a way of doing this that seems especially prone to
Comment via: facebook
I'd expect that too many people apply for most units for it to be worth that much effort to game. A quick search indicates that 40-500 people might apply for a unit. If it takes someone several years to successfully apply after they are eligible, that doubles the lost earnings. They also can't resell the unit for several decades, so they are locked in for a long time. The option value of moving in the next 40 years seems significant.
I agree that today this isn't too much of an issue because there are so few units. But there are proposals to build far more affordability-restricted housing, and I no longer think that's such a good idea.
Some of what I describe above, however, is an issue today. Consider the "Buy land, take advantage of density bonuses, build a large 100% affordable fancy building, and sell the units to your just-out-of-school currently-low-earning children" case.