I can't think of any wood I hate. I love cocobolo. The only problem is it loads belts very quickly, just a step below Lignum Vitae - fastest wood to load a belt. But cocobolo's smell and ease of drilling and sanding as well its stability makes it one of my favorite budged woods. I have several boards of it and will offer it for as long as I can obtain it.
Ebony is also my favorite, though fine dust will stain your hands and you have to wear respirator. For that matter, one should wear respirator with working any woods, including domestic hardwoods like walnut.
Poorly stabilized woods, particularly acrylic stabilized, are a huge pain to work with. Bits clog up with gooey stuff that needs to be removed immediately, or it will glue itself to warm bits and will have to be scraped. I won't work any woods stabilized by other than K&G and one or two other companies.
In terms of hardest wood to work with, I would say tiger maple and other highly figured (grain in all direction) hardwoods when carving a cavity for saya. Carving with the grain is not difficult even in hardwoods like maple, but against the grain or carving a difficult grain, could take a long time, there is tearing and chisels dull quickly. Sometimes I choose layout for the best figure, even if it is against the grain.
So in short, I don't know of any woods that I hate. I wear respirator, work clothes and take a shower after working woods like ebony.
M