How intolerant a magnet can be