ML.Detokenize¶
| Name | Mandatory | Description | Default | Type |
|---|---|---|---|---|
⬅️ Input |
The input of the shard, if any | [Int]Tensor |
||
Output ➡️ |
The resulting output of the shard | String |
||
Tokenizer |
No | The tokenizer to use for detokenization. | none |
Var(Tokenizer) |
SkipSpecialTokens |
No | If true, skip special tokens during detokenization. | true |
Bool |
Converts token IDs or tensors back into text using a tokenizer.