DOLPHIN - SMASH Benchmarks

Traditional suppliers of EDA solutions have always resisted the emergence of "evaluation benchmarks" while each customer develops at pain some custom benchmark in the absence of a yardstick for truthfulness.

Golden benchmarks for simulators focus narrowly on the speed of execution and the accuracy of computations on circuits which have already been debugged.

An appropriate benchmark demands the capability to discriminate bugs as the only relevant contribution for designer's productivity and safety in design!

Considering that 99% of designers' time, on any circuit, is spent at debugging, or even proving the absence of bugs, benchmarking simulators should firstly differentiate between a front-end activity during actual design or Virtual Test and Diagnostic, versus a back-end activity during development finishing or Virtual Characterization.

In other words benchmarking simulators should rather rely on the actual efficiency of:

Any simulator optimized for speed is the optimal choice only in the absence of bugs that also corresponds to 1% of the time to design!
Meanwhile, most suppliers and CAD managers keep ignoring this fact of life.
Two types of benchmarks should reveal the capabilities of a fully-endowed simulator:

initialization benchmarks must serve for testing the capability of its convergence heuristics for detecting operating states, namely those where the circuit remains quiescent in the absence of inputs and which serve to improve testability: to begin with stable states must be identified, metastable states must be eliminated, and one stable state must be selected as "initial state" for its features facilitating Virtual Test and Diagnostics, like being the easiest to lift indeterminacies with a short inputs sequence. The importance of this design stage must not be overlooked, as improper operating states actually are design weaknesses potentially worse than frankly deterministic bugs. Thoroughness of identification of metastable states thus is a more relevant criterion of worth than speed of convergence!
detection benchmarks must serve for testing the capabilities of its simulation algorithms to detect and locate as fast as possible, and with accuracy, all functional bugs even those appearing only as yield drops

Significant productivity improvements can be achieved at this stage, when large realms of data must be extracted, depending on how properly the simulation set-up has been prepared during the front-end validation for iterating now, both without and with parasitic extractions.
The benchmarking process then must be focused on Virtual Characterization, i.e. pre-characterizing circuits as fast as possible. At this stage the benchmark must serve to evaluate on the one hand the duration of simulations launched according to a predefined set of conditions and, on the other hand the quality of the exploited simulation results according to designers' needs such as the thoroughness and truthfulness of data sheets generated.
Too many a novice designer would use as reference a common simulator on the market, instead of comparing with measurements on actual silicon: Testchips are needed for simulator calibration!

To summarize these simple facts, SMASH thus is optimized as "front-end" simulator for detecting design bugs most efficiently with innovative features for Virtual Test and Diagnostics. This is opposed to simulators merely optimized for data-mining in "back-end" mode with mere focus on Virtual Characterization as fast as possible.
SMASH is nevertheless augmented in back-end mode with an iterator named SHAKER, where the front-end mode serves to identify the proper solution before running in back-end mode with the iterator!

In the front-end mode:

In the back-end mode: