Could you give me examples of: Parts of plants which taste fruity, but
are not fruits.
(The only example I can think of is rhubarb.)

+ + +
cashew (not the nut), juniperberry, yew berry, etc BRBR

Juniper and yew berries are fruits. They are called cones, but they contain
seeds if they are fertilized.

The fruity part of a strawberry is an achene. The actual fruits are those
little things all over the outside which contain the seeds. The edible part of
a pineapple is also a holder. Our modern pineapples are seedless. Not sure
which part you would call a fruit.
One notable example is the Japanese raisin tree, Hovenia dulcis. The fruits
don't amount to much, but when they ripen, the fruiting twigs become soft,
tasty, & edible.
Nature is varied and ingenious. Fruits evolve and are further cultivated to
enhance the reproductive ability of the plant. Any part of a plant may become a
storage organ to tide the plant over the winter or the dry season. This can be
leaves, stems, sap, or roots. If they are used to store sugar, people (and
animals) may find them tasty, and possibly fruity.
Iris,
Central NY, Zone 5a, Sunset Zone 40
"If we see light at the end of the tunnel, It's the light of the oncoming
train."
Robert Lowell (1917-1977)