Skip to content

novelty-check

Category: audit
Field: —
License: MIT
Updated: 2026-05-18
Stages: referee-simulation

Novelty Check Skill

Check whether a proposed method/idea has already been done in the literature: $ARGUMENTS

Constants

  • REVIEWER_MODEL = gpt-5.5 — Model used via Codex MCP. Must be an OpenAI model (e.g., gpt-5.5, o3, gpt-4o)

Instructions

Given a method description, systematically verify its novelty:

Phase A: Extract Key Claims

  1. Read the user's method description
  2. Identify 3-5 core technical claims that would need to be novel:
  3. What is the method?
  4. What problem does it solve?
  5. What is the mechanism?
  6. What makes it different from obvious baselines?

For EACH core claim, search using ALL available sources:

  1. Web Search (via WebSearch):
  2. Search arXiv, Google Scholar, Semantic Scholar
  3. Use specific technical terms from the claim
  4. Try at least 3 different query formulations per claim

  5. Include year filters for 2024-2026

  6. Known paper databases: Check against:

  7. ICLR 2025/2026, NeurIPS 2025, ICML 2025/2026

  8. Recent arXiv preprints (2025-2026)

  9. Read abstracts: For each potentially overlapping paper, WebFetch its abstract and related work section

Phase C: Cross-Model Verification

Call REVIEWER_MODEL via Codex MCP (mcp__codex__codex) with xhigh reasoning:

Text Only
config: {"model_reasoning_effort": "xhigh"}
Prompt should include: - The proposed method description - All papers found in Phase B - Ask: "Is this method novel? What is the closest prior work? What is the delta?"

Phase D: Novelty Report

Output a structured report:

Markdown
### Novelty Check Report

#### Proposed Method
[1-2 sentence description]

#### Core Claims
1. [Claim 1] — Novelty: HIGH/MEDIUM/LOW — Closest: [paper]
2. [Claim 2] — Novelty: HIGH/MEDIUM/LOW — Closest: [paper]
...

#### Closest Prior Work
| Paper | Year | Venue | Overlap | Key Difference |
|-------|------|-------|---------|----------------|

#### Overall Novelty Assessment
- Score: X/10
- Recommendation: PROCEED / PROCEED WITH CAUTION / ABANDON
- Key differentiator: [what makes this unique, if anything]
- Risk: [what a reviewer would cite as prior work]

#### Suggested Positioning
[How to frame the contribution to maximize novelty perception]

Important Rules

  • Be BRUTALLY honest — false novelty claims waste months of research time
  • "Applying X to Y" is NOT novel unless the application reveals surprising insights
  • Check both the method AND the experimental setting for novelty
  • If the method is not novel but the FINDING would be, say so explicitly
  • Always check the most recent 6 months of arXiv — the field moves fast
  • Anti-hallucination for Closest Prior Work. Every paper in the prior-work table must pass pre-search verification via verify_papers.py (canonical name resolved per shared-references/integration-contract.md §2; 3-layer arXiv / CrossRef / Semantic Scholar fallback inside the helper itself). Policy D1 (primary + degraded-output fallback): if the helper is unresolved or its invocation fails, tag candidate entries [UNVERIFIED] and surface the uncertainty rather than dropping them. Never fabricate arXiv IDs, DOIs, or titles from memory. Full protocol in shared-references/citation-discipline.md § Pre-Search Verification Protocol.

Review Tracing

After each mcp__codex__codex or mcp__codex__codex-reply reviewer call, save the trace following shared-references/review-tracing.md (Policy C — forensic; never silently skip). Use save_trace.sh (resolved per the chain in shared-references/integration-contract.md §2) or write files directly to .aris/traces/<skill>/<date>_run<NN>/. Respect the --- trace: parameter (default: full).