Remove the empty column when there are no input groups.