RLTasksby ulam.ai
Sign inAll tasks
Software engineeringSWE-0240

SWE-0240

A long-horizon software-engineering incident that migrates compromised browser token custody to a server-mediated OAuth/OIDC architecture. Across an 8k-line workspace, agents must diagnose cross-service failures, reason about concurrency, implement Authorization Code with PKCE, preserve migration compatibility, and recover safely under sealed generated traces and mutation-sensitive verification.

Coding agentPrivate verifierRL environment
Version3.0.0
Environments1
RewardScalar · 0–1
DeliveryPrivate ZIP
01 Task contract

A real, versioned RL environment.

A long-horizon software-engineering incident that migrates compromised browser token custody to a server-mediated OAuth/OIDC architecture. Across an 8k-line workspace, agents must diagnose cross-service failures, reason about concurrency, implement Authorization Code with PKCE, preserve migration compatibility, and recover safely under sealed generated traces and mutation-sensitive verification.

Agent objective

  • Interact with the supplied stateful environment.
  • Produce verifier-checkable actions or artifacts.
  • Maximise scalar reward under the package contract.

Evaluation

  • Private verifier.
  • Reported reward range 0–1.
  • Package-specific public and private checks.

Delivery boundary

  • Private object stored in Cloudflare R2.
  • Authenticated entitlement required.
  • Short-lived signed URL per download.
02 What you will work on

Distinct environments, one demanding research contract.

Enough detail to understand the intellectual terrain; generated instances, hidden mechanisms, and solution paths remain inside the private package.

EnvironmentMathematical or technical frontierAdaptive research problem
Token-custody migrationBrowser-to-server auth architectureRemove compromised browser-held credentials while preserving a controlled compatibility path during rollout.
OAuth/OIDC with PKCEAuthorization-code lifecycleImplement state, nonce, verifier/challenge, callback, exchange, and session binding across services.
Concurrency and replayTabs, callbacks, retries, rotationPrevent duplicate exchange, stale callbacks, fixation, and cross-tab races under generated timing traces.
Failure recoveryNetwork loss and partial migrationResume or safely abort interrupted login, refresh, logout, and server-session transitions without widening authority.
Cross-service compatibilityFrontend, backend, identity providerPreserve intended legacy behavior while tightening boundaries and satisfying mutation-sensitive private contracts.
03 Why it is interesting

What the supplied evaluation reveals.

The human reference passes all 154 sealed outcomes and eight retained replay seeds, while the compromised starter passes only 87 and receives zero reward across six critical failure classes. All 64 targeted mutants are detected. The supplied local candidate score is 0.0, so the catalogue preserves that result rather than implying success. The package is a commercial pilot candidate with open hosted-isolation, signing, legal, contamination, and independent-calibration gates.

We publish aggregate behavior and task structure, while withholding generated instances, hidden labels, exact successful probes, private checks, and solution trajectories.

04 Supplied evaluation

Observed evaluation result.

Shown with its provenance and limitations; it is not a performance guarantee.

i
Methodology matters

The supplied local score artifact records reward 0.0; displayed without reinterpretation.

Evaluated system / policyGPT-5.6 Pro

As identified by the supplied artifact.

Recorded reward0.000

1 reported runs.

Result artifactIncluded

SWE-0240X-v3-candidate-manifest.json

Public result record

Machine-readable provenance and the exact displayed metric are available in results.json.

Open result JSON
05 Private delivery

The package stays off the public website.

The paid ZIP will live in a private R2 bucket. Vercel authorizes the buyer and issues a 2–5 minute object URL; R2 serves the bytes directly.

Included with purchase

  • Exact package version 3.0.0
  • Environment and task contracts
  • Verifier or scoring interface
  • Supplied reference/evaluation artifacts
  • Purchase record and licence v1.1
delivery flow
Authenticated buyer + entitlement check
+ private R2 object + 2–5 minute signed URL
= direct, auditable download

Package SHA-256
fcfd90ef20e9f3862c54d6d298aee48feb90a1de47e1bca5b49be917865ca769
06 Licence v1.1

Commercial use, without exclusivity.

One purchase licenses this identified item to one legal organisation for worldwide, perpetual commercial model training, evaluation, research and development. Redistribution and resale of the package are not permitted.

Read the full licenceYotta Content LTD · business customers only
SWE-0240

Ready to add this environment?

Back to marketplace
Stripe checkout

Business purchase confirmation

Sign in or create an account, then complete secure Stripe Checkout. Access is granted only by the verified payment webhook.