"Yellowjackets also live in the
ground, nasty buggers, but they aren't pollinators. "

Guess again.

Yellowjackets and other wasps are pollinators just not as efficient as
the fuzzy bees at the job.

Ground nesting solitary bees are generally docile, stopping to mess
with you means their young do not survive.
You could get stung if you pinched one or put it in your pocket or
otherwise trapped it in your clothing, but that's the kind of effort
it would take.