-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The "final" ISPD-2012 Benchmark Suite was released. Please look under the ISPD-2012 Contest Benchmark and Results section for more details.
The ISPD-2012 contest presentation with results was released.
The Benchmark release notes, Contest details, Evaluation code and metrics can be found in the Download Material section.
Please cite the following paper
when you refer to
these benchmarks in a publication:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
M. M. Ozdal, C. Amin, A. Ayupov, S. Burns, G. Wilke, C. Zhuo, "The
ISPD -2012 Discrete Cell Sizing Contest and Benchmark Suite", Proc. ACM
International Symposium on Physical Design, pp. 161-164, 2012.
Thanks again for the participation in ISPD 2012 Discrete Gate Sizing
Contest!
Feb 5, 2012 (SUN): New version of evaluation code posted. A bug reported by one of the teams was fixed. Original script doesn’t update pin caps when gates are sized.
Jan 29, 2012 (SUN): Evaluation program posted. Slides and cpp code available for download on evaluation and ranking section. The updates include the evaluation code, the updated "Evaluation_Metric.pdf" and the "evaluation code release note" detailing how maximum capacitance violations are computed.
Jan 23, 2012 (MON): The hardware/software specifications for evaluation are released. Please check the submission information section for details.
Jan 8, 2012 (SUN): The library and the benchmarks are updated. The details about the changes in the library can be found in the presentation "Contest library release notes" . The new benchmark tar-file is available here. The "Benchmark release notes" slides are updated accordingly. Summary about all the released benchmarks can be found in the README file, which is part of the tar-file.
Jan. 3, 2012 (TUE): The evaluation and ranking is updated. Presentation with detailed evaluation metrics was posted on evaluation and ranking section. List of supported libraries was added in the supported libraries section.
Jan. 2, 2012 (MON): The contest schedule is updated. The final submission deadline is Feb. 16, 2012. Each team must also submit the trial version of the binary along with the agreement of "ISPD Supplemental Submission Rules" by Feb. 2nd, 2012 (the trial version submissions are only for compatibility test and hence not necessarily to be functional).
Jan. 2, 2012 (MON): Clarification - If you are using PrimeTime or another external timer, save all .tcl files (from the distribution we sent you) undere $ISPD_CONTEST_ROOT directory for proper sizer/timer interaction.
Dec. 27, 2011 (TUE): Important Reminders on Timer Usage - 1) If you are planning to call Synopsys PrimeTime during optimization (sizing), then you MUST use the mechanism provided EXACTLY. You CANNOT and WILL NOT BE ALLOWED TO call PrimeTime in any other way during optimization when we evaluate your submission. 2) We will accept ONLY binaries from you for evaluations. No TCL scripts will be accepted from you. The submitted binaries MUST work with the TCL scripts we have provided if communication with PrimeTime is needed (PrimeTime is needed if you did not create your own timer). If you are using another external timer at your end, then you MUST ensure that your submitted binary will work with PrimeTime.
Dec. 27, 2011 (TUE): Change of name of intermediate file containing cell sizes for timing calls - In order to avoid confusion with the name of the final .sizes file, we have modified the TCL and C++ API distribution for communication with PrimeTime (and any other external timers you may use). The intermediate file written out by C++ API call to timer is now called <benchmark>.int.sizes as opposed to <benchmark>.sizes to avoid confusion with the final <benchmark>.sizes file that your binary must write out. The file format still remains the same. If you have indicated that you will be using PrimeTime or another external timer, then we will separately email you the updated distributions. Also, please see the updated Contest details document. This announcement does not apply to those of you who are building your own timer.
Dec. 11, 2011 (SUN): The finalized "ISPD Supplemental Submission Rules" for the ISPD contest was emailed to the contestants. The contestants have to explicitly state the agreement with the legal statements in the form when submitting the binaries. The agreement with the statement allows the contest organizers to evaluate the submitted binaries.
Nov. 28, 2011 (MON): "parser_helper.cpp" in the benchmarks is updated to fix a bug. Please download the updated benchmark files.
Nov. 14, 2011 (MON): "Contest Details" section is updated to include the note on timer API.
Nov. 14, 2011 (MON): the contestants are recommended to frequently check the contest website for the benchmarks updates. There will be new release of benchmarks with changes in the library delay/slew tables.
Nov. 02, 2011 (WED): the benchmarks and the release notes are updated to include two more test benches. Please download the most recent benchmark and the release notes for details. The "contest details" is also updated to include the corresponding changes.
Oct. 12, 2011 (WED): "Contest Details" section is updated to include the information about the choice of timers.
Oct. 10, 2011 (MON): The website is updated with the contest details. We reserve the right to make changes in the contest rules or benchmarks in the future. It is your responsibility to check the contest announcements on the web page until the submission deadline
Oct. 10, 2011 (MON): The sample benchmarks and parsers for the benchmark files are released and can be found under "ISPD-2012 Contest Benchmark and Parsers".
Oct. 10, 2011 (MON): The detailed description about the ISPD 2012 contest is ready. Please read the file "ISPD_2012_Contest_Details" under "Contest Details". This contains the important information regarding the contest.
Sept. 27, 2011 (TUE): Call-for-participation (CFP) is ready:
Call for Participation-----------------------------------------------------------------Start Date: October 10th, 2011Registration Deadline: December 12th, 2011-----------------------------------------------------------------
More details can be found at CFP (pdf or txt).
Name | Description | Last Update |
Final contest benchmark | The benchmark tar ball for ISPD 2012 contest | Mar. 28, 2012 |
Benchmark release notes | The benchmark release information | Mar. 28, 2012 |
Evaluation code | The evaluation code | Feb. 5, 2012 |
Evaluation code release note | The release note of the evaluation code | Jan. 29, 2012 |
Evaluation metric | The evaluation metric information | Jan. 29, 2012 |
Contest library release note | The contest library release notes | Jan. 8, 2012 |
Contest details | The detailed description of the ISPD 2012 contest | Dec. 27, 2011 |
*The most recently updated files are labeled in red.
The detailed description of the ISPD 2012 contest can be found in this file: ISPD_2012_Contest_Details.pdf. The file contains the information about: (1) submission information; (2) contest evaluation; (3) benchmark files & scripts; and (4) sizer/timer interaction. Teams are advised to read this file as it has important information regarding the benchmark suite and utility scripts.
If you plan on using Synopsys PrimeTime as your timer: Please note that the contest organizers will be using their own versions of timer.tcl, pt_scripts.tcl, and shuttimer.tcl. We have already released these files to you (and may provide later updated versions to you). We will not be accepting these files from the contestants.
If you plan on using any other timer: Please note that the contest organizers will be using their own versions of timer.tcl, timer_scripts.tcl, and shuttimer.tcl. We have already released these files to you (and may provide later updated versions to you). We will not be accepting these files from the contestants.
* Other names and brands may be claimed as the property of others.
The ISPD 2012 Discrete Gate Sizing Contest uses realistic benchmarks to evaluate all the contest submissions. All benchmarks consist of the verilog netlist, constraint file, parasitics file and cell library. Please read ISPD_2012_Contest_Details.pdf for benchmarks details.
The final benchmarks and related parsers for the ISPD 2012 contest can be found here. The details about the benchmarks and changes can be found in the Contest library release note and Benchmark release notes.
*The earlier benchmark suite was released for contestants to test the algorithms and can be found here.
Please send all emails regarding contest registration to Cheng Zhuo (cheng.zhuo_at_intel.com).
Please send all emails regarding timing to Chirayu Amin (chirayu.s.amin _at_ intel com).
Please send all emails regarding benchmarks to Andrey Ayupov (andrey.ayupov _at_ intel com).
Please send all emails regarding cell library to Steven Burns (steven.m.burns _at_ intel com).
Please send all emails regarding evaluation metrics or system specifications to Gustavo Wilke (gustavo.r.wilke _at_ intel com) and Cheng Zhuo (cheng.zhuo_at_intel.com).
For any other topics, please send emails to Mustafa Ozdal (mustafa.ozdal _at_ intel com).
Please add "ISPD2012" to your subject line to get a quick response from the contest administrator!
Oct 10, 2011 | Official start date. The sample benchmarks were released. |
Dec 12, 2011 | Registration deadline |
Feb 2, 2012 | Trial version submissions for compatibility test |
Feb 16, 2012 | Receive submissions from all teams |
Mar 25-28, 2012 | ISPD 2012, announce contest results |
Please try to submit a static binary as it helps with portability.
Each team is allowed to submit a single binary that should run on all benchmarks.
The submitted work can be either single threaded or multi-threaded version.
An alpha (preliminary) binary submission is required two weeks before the final submission deadline.
No pre-computed information can be used to influence the current run. The run directory will be cleaned prior to each run.
The officially supported programming language will be C/C++. For other languages, please check with the contest organizers first. MATLAB will not be available.
The library for the contest is the standard C/C++ library. For any parallel library or third party public domain library, please check with the contest organizers first.
You are allowed to implement your own timer, but the final timing evaluation will be done using Synopsys PrimeTime*.
System Specification:
Hardware:
Microprocessor:
2 sockets of 6-core Intel Xeon X5675 (See
http://ark.intel.com/products/52577/Intel-Xeon-Processor-X5675-(12M-Cache-3_06-GHz-6_40-GTs-Intel-QPI
for details)
Memory:
96GB RAM
Software:
Linux version: 2.6.32.29-0.3-default
Gcc version: 4.3.4
PrimeTime version: F-2011.06-SP3
GNU libc version: 2.11.1
* Other names and brands may be claimed as the property of others.
There will be two separate rankings.
Primary ranking: Solution quality will be the main metric. Runtime will be used for tie-breaking.
Secondary ranking: Both solution quality and runtime will be important. Multi-core implementations are encouraged!
There will be a hard runtime limit for each benchmark.
Quality metrics in order of importance:
Timing, slew, and max-load violation. Note: we expect each benchmark to have a solution with zero violations
Cell leakage power
The detailed order will be based on:
A solution with no violations will always be better than another with some violations
If there are two solutions with zero violations, then the one with the smaller total leakage will be ranked higher
If there are two solutions both of which have non-zero violations, then the one with the smaller total violation will be ranked higher
Detailed ranking metric can be found at Evaluation_Metric.pdf. The evaluation program can be found here. Please read the release note for details. For any other questions please refer to ISPD_2012_Contest_Details.pdf.
Below is the list of C/C++ libraries are allowed and supported by the contest organizers, if you wish to use a library not listed below please contact contest organizers first. Supported libraries:
Team Name | Participating Team | Team Members |
NuTuner | Northwestern University | Li Li, Yinghai Lu, Peng Kang |
HBLR | University of Texas, Austin | Subhendu Roy, Jagmohan Singh, David Z. Pan |
National Chung Cheng University | National Chung Cheng University | Chih-Cheng Hsu, Shun-Ren Siao, Bo-Syun Jhang, Shu-Hung Lin, Jen-Hsiang Lee, Mark Po-Hung Lin |
UFRGS-BRAZIL | Universidade Federal do Rio Grande do Sul | Gracieli Posser, Guilherme Flach, Tiago Reimann, Jozeanne Belomo, Marcelo Johann, Ricardo Reis |
CUsizer | The Chinese University of Hong Kong | Tao Huang, Wing-Kai Chow, Yuan Jiang, and Evangeline F. Y. Young |
National Cheng Kung University | Jiun-Li Lin, Shun-Ming Xu, Ching-Lun Lin, Ing-Chao Lin, Tsung-Yi Ho | |
PowerValve | National Tsing Hua University, Missouri University of Science and Technology | Chi-Hsuan Lin, Chung-Han Chou, Kuan-Yu Lai, Rui-Xiang Xu, Yi-Chiao Chen, Yiyu Shi, Shih-Chieh Chang |
Nizhny Novgorod State University | Iosif Meerov, Kirill Kornyakov, Andrey Kamaev, Polina Rozenshtein, Boris Rozenshteyn, Ilya Lebedev, Lyubov Popova, Belyakov Aleksandr | |
eOPT | New Mexico State University | Mustafa Aktan, Vishal Nawathe, Vojin Oklobdzija |
SensOpt | University of California, San Diego | Seokhyeong Kang |
NTHU KHWL | National Tsing Hua University | Po-Yi Hsu, Yung-Shun Lin, Tien-Yu Kuo, Sheng-Kai Wu |
NCTU CIA | National Chiao-Tung University | Chia-Ling(Lynn) Chang, Hsuan-Ming(Ryan) Huang, Hsuan-Wei(Hunta) Wang |
NTUgs | National Taiwan University | Kuan-Hsien Ho, Po-Ya Hsu, Yu-Chen Chen, and Yao-Wen Chang |
NCTU_CIA | National Chiao-Tung University | Lin-Sheng Yan, Xu-Jia Kang, Xu-Kai Hua, Zhang-Zhe Cheng |
Team-SJ | SoftJin Technologies Pvt Ltd | Mukul C Dutt, Murali M Reddy, Ambica Prasad |
Gatekeeper | National Chiao-Tung University | Chuan-Yao Tan, Chang-Cheng Tsai, Chih-Long Chang, Hua-Yu Chang, Iris Hui-Ru Jiang, Yao-Wen Chang |
UIC Dart Lab | University of Illinois at Chicago | Ren Huan, Shantanu Dutt |
Goldilocks | University of Michigan | Myung-Chul Kim, Jin Hu, Igor Markov |
For the contest announcement and call for participation, please see pdf or txt.