'Bull - it has to be. Its where I spend the most time. And where most of my actions come from."
All that is true. However it takes a lot of work to make the structure of a piece of software designed to manage email into a system which is flexible enough to help you control all of your stuff.
A key issue is the actions which arise outside of your email - voice mails, twitter messages, physical paper stuff (remember that?), texts, messaging from online collaboration systems, whereever.
All of those items need to be incorporated into your system. And the emails which are non-action items (for review, for reference, for delegation or just to be deleted) need to be cleared out of it.
As you move from a basic email client or service (ie Mail on the Mac) to a more featured offering like Outlook then the task becomes easier with discrete areas for tasks and calendar items which are nicely integrated. Addons such as the GTD® Outlook® Add-In by Netcentrics or the Jello dashboard for Outlook.
Ultimately you have to balance the friction which comes from using multiple pieces of software with the compromises around implemention of your system using software not designed for that purpose.