It is perfectly normal for the output shaft to move around because it rotates on two roller bearings. The front bearing is a 6308N, and the rear is a 63/28.
If the output shaft was designed like the pinion in a differenial, it would use tapered Timkin bearings, which when properly adjusted, or preloaded, there would be very little or no movement. This would require additional labor and parts to set up the case, so Toyota chose the cheaper, more loose way of building the t/case.
To check the rear output flange, simply remove the rear drive shaft, and tighten the output flange nut.
You will need a 32 mm socket. Torque spects call for 95 ft lbs, and remember to restake the nut.
Reinstall the rear drive shaft and call it good.
