Tokens are not words. Modern LLMs chop text into sub-word pieces with Byte-Pair Encoding, which is why GPT-5, Claude, and Gemini handle Italian, Japanese, and Arabic almost as fluently as English.
Home