Not if the trust is prepared correctly. With most trusts, you can change your trustee at any time if you aren’t satisfied. Even with an irrevocable trust, you or your beneficiaries can have the right to change the corporate trustee.
Also, the trustee you select must follow the instructions you put in your trust — while you are living, if you become incapacitated, and after you die. That’s because a trust is a binding legal contract, and your trustee can be held liable if he or she doesn’t follow your instructions.