When I was your age… I walked to school uphill both ways.
This is something that Grandpa may have said to underscore the challenges that the younger generation does not have to endure. There are variations of this saying – I walked for miles to school, in the snow, uphill both ways; back in my day I walked uphill both ways to school in freezing temperatures and snow!

During the days of one-room country schools, children walked one to two miles to their schoolhouse. The schools were spaced throughout the townships so that no child had to travel more than two miles to get from home to school, by foot, horse, or automobile.
The consolidation of schools brought the demise of DeKalb County’s one-room schools based on the decline of the rural population and growing cities.
Source: “Rural School Journeys: A Legacy of Learning” by the DeKalb County Historical-Genealogical Society