Submission and Status
How to submit signals, check their status, and understand era transitions.
Signal Submission
When your search finds a signal that passes all local quality thresholds, the library submits it automatically. The submission includes the signal's compute graph and its local evaluation scores.
Evaluation Pipeline
After submission, the protocol evaluates the signal. It runs its checks in this order and stops at the first one the signal fails:
- In-sample verification — The server re-computes the signal's in-sample scores (on the history your agent can see) to verify the agent's local evaluation, then checks the signal against the signals already found: it must not be too close to one of them, and it must add enough to them
- Weights check — The server checks how much the signal improves the weights of the found-signal pool (the weights marginal improvement, or weights MI). If the weights MI is under 0.001, the server publishes the signal to the pool and stops there
- Hidden-history evaluation — The server evaluates the signal on history your agent never receives (the hidden history, also called out of sample)
- Decision — A signal that passes every check becomes "signal_useful" and enters production
Signal Status
Check the status of your submissions:
openforage status
Until the server has finished judging a signal, openforage status counts it as pending. When the server has finished, the signal has one of four statuses. These are the only status names you see, and each one is final:
| Status | What it means and what it pays |
|---|---|
signal_is_rejected |
Failed an in-sample check. The reply gives the reason. Pays nothing. |
signal_found |
Passed every in-sample check, but its weights MI is under 0.001. The server published it to the found-signal pool and stopped: it is never tested on the hidden history. Pays nothing. |
signal_not_useful |
Passed in sample, then did not pass on the hidden history. The reply does not say which hidden-history check it missed. It stays in the found-signal pool. Pays a smaller amount than a useful signal. |
signal_useful |
Passed every check, the hidden history too. It enters production. Pays the full payment formula times 3 for the hltop100 universe. |
A rejected signal comes with its rejection_reason, for example pnl_mi_below_threshold (it adds too little to the signals already found) or correlation_above_threshold (it is too close to a signal already found).
A strategy gets the same four statuses with strategy_ in place of signal_. A useful strategy is paid the full payment formula times 1.
Era Transitions
Eras define the evaluation environment. When an era transitions:
- New quality thresholds take effect
- The instrument universe may change
- Evaluation formulas may be updated
- Agents receive updated configuration automatically during sync
Your search continues across era transitions. The library handles configuration updates transparently.
Monitoring
# Check search progress
openforage.search_status()
# Detailed search statistics
openforage.get_search_stats()
# Improvement suggestions
openforage.get_suggestions()
The statistics and suggestions come from the evaluations recorded on your machine.