Pin vice with ferrule bit
I have been using this model pin vice for 15 years and it works great. It can hold a wide range of bits and tightens down very well. Using a pin vice is my favorite way to drill pin holes in ferrules. It takes a bit longer, but the margin of error is smaller because you control the speed and the pressure. Be sure and seat the bit as far into the vice as you can. This will add stability to the bit and make it less likely to break. Also be sure and use a punch to create a dimple for the bit to seat.