Remove obsolete segment skip checks from tokenization.

Change-Id: Iac7bc0d0eba459a04688aae224d34ae9b59742db
3 files changed