I actually started wheeling before I was born and again hit the trails soon after I was born. The issue with the Moab trip is that it's 1,100 miles each way on the road and it's only worth making the trip if I can get 4 or 5 full days on the trail. I have no problem taking a little one on the trails, just not too sure about a week long trip.
My wife really wants to drive Hell's Revenge so we may look for some grandparent babysitting services for a trim in a couple of years.