shipd-olympus-skill

Appearance

Submission workflow ​

The end-to-end path, in plain terms. Each stage gates the next — failing early is cheap, failing late costs platform tokens.

1. Pick an eligible repo ​

Four hard requirements: a permissive license (MIT/Apache/BSD — open the LICENSE file yourself, GitHub's API field is often wrong), 500+ stars, a commit within the last 12 months, and an allowed language (TS/JS/Python/Go/Rust/C++/Java). Paste the exact repo URL into the platform form — its search box surfaces zero-star forks, and some repos are silently blocked.

2. Design a task agents will fail ​

This is the stage that decides everything. The platform only accepts tasks where top AI agents fail at least half their attempts — so difficulty is designed in, not discovered later.

The recipe: find a complex subsystem in the repo (a parser, a type dispatcher, a scheduler), then design a small feature family that has to integrate with it correctly. Make it hard with precision traps, not volume:

  • Convention traps — your spec picks one of several textbook conventions (quantile interpolation alone has ~9); agents implement a neighboring one and fail.
  • Exact-boundary rules — one sentence with several ways to get it subtly wrong.
  • Deliberate differences from the sibling function — agents copy the existing function's semantics and miss your stated substitution.
  • Wide wiring — the feature must register in many places; forgetting one fails a test. This also lifts the solution-size medians the platform checks.

Avoid anything needing a browser, clipboard, audio, or network — the test container runs fully offline. Avoid mega-popular quest-farm repos (similarity-check collisions). And ambiguity is not difficulty: a 0% pass rate usually means one misreadable sentence, not a hard task.

3. Sweep for originality ​

A PR in any state — open, merged, or closed — that implements your idea kills the submission. It's the #1 rejection reason. Sweep PRs, issues, and GitHub Discussions for every function name you plan to add. Then find the commit that added the closest analogous feature: its file list is your wiring checklist.

4. Write the description ​

Write it like a maintainer filing an issue: prose, opens with the ask, no headings or code or file names. Define every function you name. Keep the sentences your hidden tests depend on — the deliberate substitutions and boundary rules — and let the AI checker trim everything an implementer could discover from the repo.

5. Write the hidden tests ​

Random-hash filenames (openssl rand -hex 3) so an agent's own test files can't collide with yours. Every test must fail on the unmodified repo — throws-tests need an "assert the function exists" first line. Never pin error messages, and never pin an exact return type (like a Fraction) unless the repo's sibling functions already demonstrate it in their own tests — that's the fairness bar.

6. Write the Dockerfile ​

Language base image, WORKDIR /app, all dependencies installed at build time, no tests in the build. JS/TS: the base image sets NODE_ENV=production, so npm ci needs --include=dev or the test runner won't exist offline. Pin any ad-hoc installs to exact versions.

7. Verify everything locally ​

Fresh clone → docker build → run with --network none → apply the test patch (base suite passes, new tests all fail) → apply the solution (everything passes) → scan patches for leak words. Never build from a working tree that has node_modules or build artifacts in it.

8. Platform endgame ​

Read the similarity-check matches yourself — a same task reworded means pivot, not reword. Check the "Submission criteria" panel for the current bars (pass rate ≤50%, ≥6 agent runs, median solution size ≥150 lines across ≥2 files). Run a small agent batch first: 100% pass means harden with another trap from stage 2; 0% means read the failing logs — you probably have an ambiguous sentence, not a hard task.