Grub dual boot error

Sounds like the grub boot block got installed to the internal hard disk instead of the external hd. I'm guessing you'd like the PC without the external hd simply boot windows, and when the external hd is attached, you get the choice of Ubuntu (default) or windows. To fix things, you need to do two things: 1)fix the Maser Boot Record (MBR) on the internal hd, and 2)install grub with all its files to the external hd. For 1) you can either use the windows install media to run a recovery and run fixmbr from a command shell. Alternatively, if you don't have the media (vendor recovery media may not work for this) and cannot borrow it, you can install grub properly to the internal hd IF you have a place for the grub files (NOT an NTFS format partition). Such a place is typically a small FAT partition used for recovery, and there is usually enough space for the grub files. On such a FAT partition, make a directory named boot, and you can then use your usb install media to install grub to the hd, specifying the location of the boot dir you made.
For 2), you should be able to just run the live usb, and install to the external hd. Do this before 1), since it is easy, and you will be able to boot Windows if you need to from the fixed external hd. Then you can fix the internal hd at your leisure (or decide you don't need to ; ).