compileCategory - What options are available for the 'tokenizer' parameter and what is the default?

Réponse

The 'tokenizer' parameter specifies which tokenizer to use. The default is 'STANDARD', which applies a language-specific tokenizer. The alternative is 'BASIC', which separates words by white spaces and punctuation, and is available for Chinese, Japanese, and Korean to enhance rule matching.
Did this answer help you?
compileCategory

textRuleDevelop

Doc : compileCategory
Other questions about compileCategory
What are the required parameters for the compileCategory action?
Lire la réponse
What does the 'casOut' parameter represent?
Lire la réponse
Can I use an existing concept model (LI binary) when compiling a category model?
Lire la réponse
What is the primary function of the compileCategory action?
Lire la réponse
What is the purpose of the 'language' parameter?
Lire la réponse
Copyright © WeAreCAS 2014-2026
SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. WeAreCAS is an independent community site and is not affiliated with SAS Institute Inc.