I had to battle the hotshot trainer to the right of the grand hall with the treeko and grovyle 3 times in a row. After the initial battle I took one step away from him but still along his eye-line and he 're-spotted' me. After that battle the same thing happened again but he has not challenged me since.