Is it simply upon hitting a certain number?
Or perhaps before that?

We grow old when we think the way we see the world is the right one.
When we believe it is our point of view that matters, and it need not be changed.
When we refuse to see another way to live life, because it will challenge our comfort zone. 

That is when we truly get old.

We grow old when we refuse to embrace the new.