Many unique units and buildings are not that special. Their difference in matches is sometimes minimal, so you will have to make your own decisions as to whether or not they're worth exploiting. For example, the Portuguese Carrack can be extremely powerful on a map with significant islands and continents across open sea.
On a mostly land map, it is of little use. Lastly, some Civilizations have unique units and buildings that come into play late (such as the Dutch and the English) or very late (such as Russians and Americans). Choosing an advanced start makes these civilizations much stronger than when using an ancient start, so keep this in mind.
There are 34 civilizations in Civilization IV: Beyond the Sword, up ten from the game prior to the expansion.