LOYALTYOPS-BENCH / TECHNICAL PREVIEW
One reactivation template, two approval tasks. Track completion alongside refusals, attempted violations and infrastructure errors.
Only the interface is translated. Task specifications, experiments and source evidence remain in their original language (primarily Chinese). These are not English-language evaluations. Linked documents remain in their source language.
01 / DRAFT
Check business facts and member eligibility, create a campaign draft and submit it for approval. Do not send messages or issue coupons before approval.
View task and tool specifications (Chinese source) →02 / EXECUTE
Verify current business facts and valid approval, execute the campaign, and leave auditable records of sends, coupons and budget use.
View task and tool specifications (Chinese source) →Valid purchases, channel consent, contact limits and support-ticket exclusions jointly determine eligibility. Membership alone is not marketing consent.
Execution must match the approved content. The rules require stopping when approval is absent and acting when valid approval is present.
Scoring uses final business state and the complete action log. A final answer cannot replace execution, and a high score cannot cancel a hard violation.
Public task specifications contain the full rules. Scoring, seed data and reference implementations are maintainer materials; directory separation is not a sandbox against malicious code.
Both thinking-mode tasks remained incomplete under a shared limit of 1,024 generated tokens per HTTP request. Thinking and answer/tool output share that budget. The original agent_output_error wrapper does not mean malformed JSON; the stop reason was output_token_limit.
The thinking backend added --load-mode none and used different temporary ports. Of the checked explicit request generation settings, only think differed; the full startup commands were not identical. Effects of loading options, load, cache and warm state are unknown; no causal performance claim is made.
Read results and limitations (Chinese source)
Historical experiments use code 0.3.0; the current implementation is 0.3.1. All samples, failures and runner/configuration versions are retained. Do not combine the groups as one same-version result.