That sounds like the right approach, adjust your topsoil to make
a decent grade. I'd be more concerned with the broken rock or
stuff, than I would be with the clay.

The broken rock may inhibit root growth and increase the soil
temperature in that area, resulting in a blighted look during
hot weather. Can you remove the rest of the rock before adding
your soil?

I was hoping not to do that. The amount of 'decorative' rubble I
removed (2-3") is sitting in a massive pile next to my front gate. It
is the same material as my driveway, so I may do a "Great Escape" and
spread it around there.
The other tarmac (US=ashphalt?) and hardcore I removed equals some
20-30 bags. Hard work to remove, and then I have to dispose of it.
I would estimate at least this amount remaining, and was hoping to
save some effort by just putting topsoil on top.
Since I live in England, hot temperatures do not worry me! Although a
piece of lawn adjacent to my rocky area does look a little hay
coloured at times, I had put it down to a sensitive ornamental grass
that colonised / took over that area. It is a fine grass, and seems
to green up if I douse it. Maybe it also has the tarmac/hardcore
underneath.
The clay does hold water. I have dug deeper for a flower bed, down to
the clay. When it rained, puddles remained in my trench.

Bob