Here are some testing strategies that I’d like to see implemented for IcedTea. I’ve asked one of our interns to investigate these points.
Reduce Mauve-on-IcedTea failures to 0
Provide reproducible way of running Mauve on IcedTea which should result in 0 failures
“make dist” target for Mauve
Once these are complete we can a) set up nightly regression runs on http://icedtea.classpath.org, b) insist on having no regressions in package releases by running Mauve as part of the RPM build.
Make IcedTea demos build standalone
The IcedTea demos include libraries built during the IcedTea build, but these libraries cannot be built using the provided sample Makefiles. Installing the java-1.7.0-icedtea-demo RPM should a) install the demo source code in /usr/share/java-1.7.0-icedtea/demo, b) install the demo code’s build requirements as dependencies, c) provide working sample Makefiles. That way, trying the demos can be reduced to something like:
yum install java-1.7.0-icedtea-demo
cp -r /usr/share/java-1.7.0-icedtea/demo ~
cd ~/demo
make -f
make -f
Ideally we’d test building and running the demos as part of the RPM build.
Run DaCapo benchmark reproducibly on IcedTea
Once DaCapo runs reliably we can set up nightly benchmark runs on http://icedtea.classpath.org.
Run graphical demos with java.awt.Robot and Xvfb
I’d like to include such graphical tests in the RPM build if the setup isn’t too fragile.
Longer term if Sun releases the jtreg harness (and some tests), we won’t hate it. 😉 More likely we’ll want to integrate it too into our RPM builds.