I tried giving myself diamond armor/weapons just to see what a difference it made... and the answer was - A LOT.

I'm going to discard that stuff and start from scratch. One item that has made a lot of difference for me is the bow (and arrows.) The ability to pick off mobs at a distance from my castle walls has increased my joy eleven-fold.