The plant is about a foot tall, with a bright yellow flower that looks
like a large buttercup. Blooms mid-to-late spring. I bought it at a
native plant sale (I'm near Baltimore), so it's native to this area.
It spread nicely.

Does it looks like the images at
http://en.wikipedia.org/wiki/Oenothera ? The most distinctive thing
here is the cross-shaped stigma (may or may not show up well in photos
due to lighting, but it is pretty easy to spot in person).

Seems to fit all your descriptions (well, with the possible exception
of native, but most/many of the yellow evening primrose species are
native to the Baltimore area).

If this is it, look closely at where your plants had been growing.
This time of year, the flower stalks are dying (or seem to be), but
the plant is putting out small clusters of leaves (from slightly
underground growths).

The drought didn't get mine (near Washington, DC), so it is possible
that you just don't recognize the flower stalk and the small leaf
clusters as being the same plant (I know I didn't the first year).