By adding the buttons to the default.config and setting the CodeViewTemplateItemList property, I was able to make the buttons appear.
However, now I'm getting some very strange behavior. If I boldface some words in HTML mode, it will make the text itself bold; it will not add tags. If I then toggle to "Normal" mode, the text is not boldfaced anymore. If I then switch back to HTML mode, the text isn't boldfaced and there's no boldface tags around the text.
Is there a way to get the boldface/underline/italic buttons to insert tags in HTML mode?