If the issue was cost, you would build schools. A better educated population get less sick, earns more and thus pays more taxes, commit way less crimes, get less social welfare, in short it is a net gain in tax dollars.
Plus, someone who gets out of jail with a big debt will very likely cost way more to society than what could ever be recovered from them.