Good flowering last year but for some reason very poor this year.

I've previously dutifully waited until after flowering before pruning
but wonder if I can start to cut back now, early, particularly as
there are almost no flowers to sacrifice.

I'm trying to promote new growth closer to the wall since this plant
is inclined to push out and overhang, stealing light from shrubs
underneath it. My expectation is that I can get a head start on that
new growth if I tackle this now.

Can anyone envisage any adverse consequences?