You need to use the Cameron-Martin-Girsanov (CMG) Theorem from Chapter 15 Section 1. The CMG Theorem essentially tells us that the relationship between the standard Brownian motion (SBM) under the real-world "P" probabilities, Z(t), and the SBM under the risk-neutral "Q" probabilities, Z~(t), is:
Z~(t) = Z(t) + gamma*t
or equally:
dZ~(t) = dZ(t) + gamma*dt
where gamma is the market price of risk, which in the context of term structure models is:
gamma = [m(t,T) - r(t)] / S(t,T)
- see page 12 of Chapter 17.
So, all you need to do is replace dZ(t) in the SDE for r(t) under P given in the question with:
dZ(t) = dZ~(t) - gamma*dt
to get the corresponding SDE for r(t) under Q.
In this case:
gamma = mu*r(t)/sigma
and you obtain:
dr(t) = sigma*dZ~(t) under Q.