One of the most annoying UI things about GA is that to get rid of a single segment is easy. Click, drag out, release. The delay between doing that and it refreshing means doing it four times is a pain. What's more you can't remove the last one.
Feature request: A single-click icon next to the segments to clear them all and return to the All Users segment only.
Customer support service by UserEcho