A prolonged period of political instability could hit Italy after voters rejected traditional parties in Sunday's election and flocked to anti-establishment and far-right groups.