no not as long as you arent in mud, thats why they should be on the back so any mud or dirt gets pushed off instead of sitting on top and causing undue wear
i wouldnt worry about it, just remember the next time you have the knuckles off or plan on any maintainance, new bearings whatever
i wouldnt take it apart just to put them back on the right side