There doesn't need to be more to it than that. The observable universe is over 93 billion light years in diameter. That means even at the speed of light, it would take over 6.5x longer than the universe has even existed for anything to cross that distance... except the universe is expanding faster than the speed of light, so actually you need to go significantly faster than light to make it across. FTL is, sadly, still firmly in the realm of science fiction, so to the best of our current knowledge most of the universe is permanently inaccessible.