Evaluation and Scoring

July, 10 2014

The Early Bird Award

All systems that have been submitted until July 6th and that don‘t have a runtime error during evaluation are eligible. The systems will be scored by their Sharpe Ratio of July: The first data point is July 1st, the last will be July 31st. You can see how well your system does, if you click on the chart and limit the time to July 1st – now. The system with the highest Sharpe Ratio in July wins $ 1,000.

Please note

Systems, that have a total runtime of 3 minutes or more only show the July performance, and not the whole backtest in the chart. The column 'performance since upload' in the Award Ranklist is just for your information. The score is shown in Column 1 as 'Sharpe Ratio (July) '. The Scores will change a lot as new market data arrives, especially in the first days. This is not a bug, it is a consequence of the definition of the Sharpe Ratio: Performance / StandardDeviaton. As only very few days influence the total performance in the beginning, the SR can both be very high and very low. As more data arrives the SR will converge to more realistic values.

The kickoff competition

Only systems that respect all aspects of the rules are eligible for the grand prize. Here is a summary of the most important rules.

For more detailed information please see the contest rules:

  • The system has a runtime on our servers of 3 minutes or less in total (rule 7.k).
  • The Trading Systems must use quantitative rules only. This includes, that Trading Systems have to be invariant to a change of the market order or a change of the market identifers. E.g. buy Apple is an invalid rule. (7.g) To rephrase that rule:
  • The system must return the same result on the contest set as on a test set. The test set has the same stocks, but in a different order. Discretionary stock-picking (buy Apple) is prohibited.
  • The system also must return the same result, if the date vector is shifted by an arbitrary number of days. This also prohibits stock picking, since it is not possible to trade the stock with a volume of 152222000 on 20030102 (which is Apple) and ignore the rest.
  • Implicitly this also means, that if the system is run twice, it has to return the same result in both runs.

The reasoning behind these rules

Ex-post stock picking does not require skill. But writing an algorithm to detect the best stocks does. The systems that qualify for the competition are also those, that will be most interesting for professional investors, and thus might earn you some real money: Investors usually are not interested in systems that are overfitted to the (known) history, just picking the best performing stock of the last 10 years, or whose success largely depends on a random seed (or dice roll).