Product LaunchSimon Willison

ttok 1.0

#tokenization#gpt-5#gpt-6#ai#release

English

Simon Willison announced the release of ttok 1.0, which counts and truncates text based on tokens. The update corrects the default tokenizer from GPT-4 to GPT-5/GPT-6, although OpenAI has not confirmed if GPT-6 uses the same tokenizer as GPT-5.

中文

西蒙·威利森宣布发布ttok 1.0，该版本基于令牌计数和截断文本。此更新将默认令牌器从GPT-4更正为GPT-5/GPT-6，尽管OpenAI尚未确认GPT-6是否使用与GPT-5相同的令牌器。