Yeovil Town manager Gary Johnson cut short Ishmael Miller's loan spell after his top-scorer vowed never to play for him again.