Many thanks to Rob Sanders for pointing this one out.
If you're a road warrior like me and you aren't always connected to a Team Foundation Server then you probably want to stop Team Explorer's default behaviour of trying to connect to TFS when starting Visual Studio. Waiting for Team Explorer to time out before realising that you really aren't connected to a network can be extremely annoying.
So whip out regedit and go do the following:
Add a DWORD value called AutoLoadServer under HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\8.0\TeamFoundation and set the value to 0.
This will fix the behaviour in Visual Studio 2005. For Visual Studio 2008 use the ...\VisualStudio\9.0... keys instead.
Yay! Freedom at last! :-)