Step downhill, over and over, and hope you picked a step size that doesn't throw you off the mountain.
On the narrow valley the tipping point is η = 0.25. Try 0.18, then 0.26.
Nudge the step size past the tipping point and watch it blow up.
This is how essentially every neural network is trained, and it is nothing more than the gradient. You are somewhere on a landscape, you want to get low, so you compute which way is steepest uphill and take a step the other way. Then again. That's the entire algorithm.
The only real decision is how big a step. Too small and you crawl. Too big and you overshoot the valley floor, land higher up the opposite wall, overshoot harder coming back, and spiral outward until the numbers blow up. On the narrow valley here the boundary is exactly η = 0.25, because that landscape's steepest direction has curvature 8 and the threshold is 2 divided by it. Set 0.18 and watch it zig-zag in; set 0.26 and watch it fly apart.
That zig-zag is worth staring at. The valley is much steeper across than along, so the gradient points mostly at the far wall rather than down the valley — and you make painfully slow progress in the direction you actually care about. It is the single most common failure mode in optimization, and it comes straight out of the second derivatives.
Momentum is the standard fix. Keep a running average of your recent steps instead of using only the current gradient: the side-to-side components alternate in sign and cancel, while the small consistent push along the valley accumulates. Turn β up on the narrow valley and watch the zig-zag straighten out.