STANDARD PROCEDURAL DATABASES, by Eric Haines, 3D/Eye, Inc.
[Created while under contract to Hewlett-Packard FSD and HP Laboratories]
Version 3.6, as of 4/3/95
address: 3D/Eye, Inc., 1050 Craft Road, Ithaca, NY 14850
email: erich@eye.com
This software package is not copyrighted and can be used freely.
History
-------
Versions 1.0 to 1.5 released February to June, 1987 for testing.
Version 2.0 released July, 1987.
Version 2.1 released August, 1987 - corrected info on speed of the HP 320,
other minor changes to README.
Version 2.2 released November, 1987 - shortened file names to <=12 characters,
procedure names to <= 32 characters, and ensured that all lines are <= 80
characters (including return character).
Version 2.3 released March, 1988 - corrected gears.c to avoid interpenetration,
corrected and added further instructions and global statistics for ray
tracing to README.
Version 2.4 released May, 1988 - fixed hashing function for mountain.c.
Version 2.5 released October, 1988 - added NFF documentation.
Version 2.6 released August, 1989 - lib_output_cylcone fix (start_norm.w was
not initialized).
Version 2.7 released July, 1990 - comment correction in lib.c, NFF file
clarifications.
Version 3.0 released October, 1990 - added teapot.c database, mountain.c
changed to mount.c, additions to def.h and lib.c, changes to README and
NFF, added patchlevel.h file.
Version 3.1 released November, 1992 - minor typo fixes, updated FTP list,
makefile update
Version 3.1a released November, 1993 - readnff added, lib split into multiple
smaller files, mac code updated
Version 3.2 released May, 1994 - added RIB output, readdxf added, more updates
Version 3.3 released June, 1994 - added DXF output, more lib splitting, etc
Version 3.3f4 released August, 1994 - fixes to Mac files (see README.MAC),
error handling for Mac added to readnff program.
Version 3.4 released October, 1994 - Larry Gritz fixed RenderMan output.
Version 3.5 released November, 1994 - Alexander Enzmann fixed PLG output and
added Wavefront OBJ output and RenderWare RWX output, and Wavefront OBJ
reader "readobj". Deleted Mac binary files from distribution (too much
trouble, and a Mac sit version is distributed separately).
Version 3.6 released April, 1995 - vertex order fixes by Enzmann and Michael
Jones, Enzmann added jacks and transformation support, other minor cleanup.
drv_hp.c changed to reflect changes to view transform output.
Version 3.7 - typo fix in NEW_HASH area of mount.c
{These files use tab characters worth 8 spaces}
Introduction
------------
This software is meant to act as a set of basic test images for ray
tracing algorithms. The programs generate databases of objects which are
fairly familiar and "standard" to the graphics community, such as the teapot,
a fractal mountain, a tree, a recursively built tetrahedral structure, etc. I
originally created them for my own testing of ray tracing efficiency schemes.
Since their first release other researchers have used them to test new
algorithms. In this way, research on algorithmic improvements can be compared
in a more standardized way. If one researcher ray-traces a car, another a
tree, the question arises, "How many cars to the tree?" With these databases
we may be comparing oranges and apples ("how many hypercubes to a timeshared
VAX?"), but it's better than comparing oranges and orangutans. In addition,
this document outlines some statistics that are more meaningful to researchers
than raw timing tests. Using these statistics along with the same scenes
allows us to compare results in a more meaningful way.
With the development and release of the Anderson benchmarks for graphics
hardware, the use of the SPD package for hardware testing is somewhat
redundant. Therefore I have deleted references to testing hidden-surface
algorithms in this version. However, another interesting use for the SPD has
been noted: debugging. By comparing the images and the statistics with the
output of your own ray tracer, you can detect program errors. For example,
"mount" is useful for checking if refraction rays are generated correctly, and
"balls" can check for the correctness of eye and reflection rays.
The images for these databases and other information about them can be
found in "A Proposal for Standard Graphics Environments," IEEE Computer
Graphics and Applications, vol. 7, no. 11, November 1987, pp. 3-5. See
IEEE CG&A, vol. 8, no. 1, January 1988, p. 18 for the correct image of the
tree database (the only difference is that the sky is blue, not orange). The
teapot database was added later, and consists of a shiny teapot on a shiny
checkerboard.
The SPD package is available via anonymous FTP from:
[ftp.]princeton.edu:/pub/Graphics/SPD
avalon.chinalake.navy.mil:/pub/utils/SPD
wuarchive.wustl.edu:/graphics/graphics/SPD (I think! Hard to get on!)
among others. For those without FTP access, write to the netlib automatic
mailer: research!netlib and netlib@ornl.gov are the sites. Send a one line
message "send index" for more information, or "send haines from graphics" for
the latest version of the SPD package.
File Structure
--------------
Various different procedural database generators are included. The
original programs were: balls, gears, mount, rings, tetra, and tree, and so
these are more commonly used for performance testing. The teapot generator
was added due to popular request (what with it being the sixth platonic solid
and all). These models were designed to span a fair range of primitives,
modeling structures, lighting and surface parameters, background conditions,
and other factors. There are also shell and lattice generators just for
amusement, and a sombrero height field generator to show off this type of
output (which is not a part of NFF, but is useful for other types of output).
A complexity factor is provided within each program to control the size of the
database generated. See the MANIFEST for a description of each file.
The compiled and linked programs will output a database in ASCII to stdout
containing viewing parameters, lighting conditions, material properties, and
geometric information. The data format is called the 'neutral file format'
(or NFF) and is described in the `NFF' file. This format is meant to be
minimal and easy to attach to a user-written filter program which will convert
the output into a file format of your choosing.
Either of two sets of primitives can be selected for output. If
OUTPUT_FORMAT is defined as OUTPUT_CURVES, the primitives are spheres, cones,
cylinders, and polygons. If OUTPUT_FORMAT is set to OUTPUT_PATCHES, the
primitives are output as polygonal patches and polygons (i.e. all other
primitives are polygonalized). In this case OUTPUT_RESOLUTION is used to set
the amount of polygonalization of non-polygonal primitives. In general,
OUTPUT_CURVES is used for ray-trace timing tests, and OUTPUT_PATCHES for
polygon-based algorithm timings. Note that sphere primitives are not
polygonalized using the simple longitude/latitude tessellation, but rather are
chopped along the six faces of a cube projected onto the sphere. This
tessellation avoids generation of the cusp points at the sphere's poles and so
eliminates discontinuities due to them.
The size factor is used to control the overall size of the database.
Default values have been chosen such that the maximum number of primitives
less than 10,000 is output. One purpose of the size factor is to avoid
limiting the uses of these databases. Depending on the research being done
and the computing facilities available, a larger or smaller number of
primitives may be desired. The size factor can also be used to show how an
algorithm's time changes as the complexity increases.
To generate the databases, simply type the name of the database and direct
the output as desired, e.g. "balls > balls.nff" creates the default sized
database and sends the output to balls.nff. A new feature in 3.0 is that you
can enter the size factor on the command line, e.g. "balls 2 > balls.nff"
gives a much smaller database of 91 spheres. You can also decide to output in
true curved or polygonal mesh modes, and can also choose from a variety of
output modes other than NFF. Typing "balls -?" gives a summary. See the
header of the database C file (e.g. "balls.c") for how the size factor
affects the output.
Other parameters in the code itself (for example, branching angles for
"tree.c" and the fractal dimension in "mount.c") are included for your own
enjoyment, and so normally should not be changed if the database is used for
timing tests. Because the hashing function in the original release of
"mount.c" is not very good (at low resolutions there is patterning), there is
a better hashing function provided which can be turned on by defining
NEW_HASH. Use the old hashing function (i.e. don't change anything) for
consistency with previously published results, use the new one for better
results.
Since the SPD package is designed to test efficiency, the actual shading
of the images is mostly irrelevant. All that matters is that reflective
surfaces spawn reflection rays and transmitters spawn refraction rays. For
this reason the effect of the other shading parameters is up to the
implementer. Note that light intensities, ambient components, etc. are not
specified. These may be set however you prefer. Feel free to change any
colors you wish (especially the garish colors of `rings'). The thrust of
these databases is the testing of rendering speeds, and so the actual color
should not affect these calculations. An ambient component should be used for
all objects, so that the time to compute it is included in the ray tracing
statistics. A simple formula for a relative intensity (i.e. between 0 and
1) for each light and for the ambient component is the following:
sqrt(# lights) / (# lights * 2).
If you desire a model for a good scene description language, NFF is not
it. Instead, you should look at Craig Kolb's ray-tracer language, which is
part of his excellent RayShade ray tracer, available via anonymous FTP from
princeton.edu:/pub/Graphics, or at POV-Ray, at ftp.povray.org. Of the public
domain ray tracers, Rayshade is the one to beat for speed, POV for popularity
(especially on the PC).
The programs all attempt to minimize program size (for ease in
distributing) and memory usage, and none allocate any dynamic memory. As
such, many of the programs are relatively inefficient, regenerating various
data again and again instead of saving intermediate results. This behavior
was felt to be unimportant, since generating the data is a one-time affair
which normally takes much less time than actually rendering the scene.
Database Analysis
-----------------
The databases were designed with the idea of diversity in mind. The
variables considered important were the amount of background visible, the
number of lights, the distribution of sizes of objects, the amount of
reflection and refraction, and the depth complexity (how many objects a ray
from the eye passes through).
balls: This database is sometimes called "sphereflake", as it is generated
like a snowflake curve. This database consists mostly of various sized
spheres. It has no eye rays which hit the background, and the three light
sources cause a large number of shadow rays to be generated.
gears: This database consists of a set of meshed gears. Some of the gears
are transmitters, making this database lengthy to render. The gear faces
each have 144 vertices, and thus tests polygon inside/outside test
efficiency. Depth complexity is medium.
mount: The fractal mountain generator is derived from Loren Carpenter's
method, and the composition with the four glass spheres is inspired by
Peter Watterberg's work. Most objects are tiny (i.e. fractal facets),
but rendering time is dominated by the rendering of the four large
spheres. Depth complexity is low, and there is much background area.
rings: Objects with six pentagonal rings form a pyramid against a background
polygon. With a high amount of interreflection and shadowing, this scene
is fairly lengthy to render. Depth complexity is also high, with
all of the objects partially or fully obscured.
teapot: The famous teapot on a checkerboard. There are a number of
variations on the teapot database, i.e. whether the bottom is included
(the IEEE CG&A article added a bottom), variations in the control points
on the lid (which creates thin, almost degenerate triangles), etc. For
this database generator, the bottom is created, and the degenerate
polygonal patches at the center of the lid and bottom are not output. The
bottom of the teapot is not flat, interestingly enough. The resolution of
the checkerboard shows the resolution of the teapot meshing (with each
teapot quadrilateral cut into two triangles), e.g. an 8x8 checkerboard
means that 8x8x2 triangles are generated per patch. The definitions for
the 32 Bezier patches are a part of the program. All objects are
reflective and there are two light sources. Depth complexity is low.
tetra: A recursive tetrahedral pyramid, first visualized by Benoit Mandelbrot
and Alan Norton. This scene is dominated by background (around 80%). With
the objects not being reflective and there being only one light source,
this database is particularly quick to render, with various forms of
coherency being very useful. Depth complexity is medium, though some
rays must pass by many triangles for some of the background pixels.
tree: A tree formed using Aono and Kunii's tree generation method. With
seven light sources, the emphasis is on shadow testing. Shadow caching
yields little improvement due to the narrow primitives, and many shadow
rays pass through complex areas without hitting any objects. There is a
fair amount of background area. Depth complexity is low.
balls gears mount rings
----- ----- ----- -----
primitives SP P PS YSP
total prim. 7382 9345 8196 8401
poly/patches 1417K 9345 8960 874K
lights 3 5 1 3
background 0% 7% 34% 0%
specular yes yes yes yes
transmitter no yes yes no
eye hit rays 263169 245086 173125 263169
reflect rays 175095 304643 354769 315236
refract rays 0 207564 354769 0
shadow rays 954368 2246955 412922 1085002
teapot tetra tree
------ ----- ----
primitives TP P OSP
total prim. 9264 4096 8191
poly/patches 9264 4096 852K
lights 2 1 7
background 39% 81% 35%
reflector yes no no
transmitter no no no
eye hit rays 161120 49788 169836
reflect rays 225248 0 0
refract rays 0 0 0
shadow rays 407656 46112 1097419
"primitives" are S=sphere, P=polygon, T=polygonal patch, Y=cylinder,
O=cone, listed from most in database to least.
"total prim." is the total number of ray-tracing primitives (polygons,
spheres, cylinders and cones) in the scene. The number of polygons and vectors
generated is a function of the OUTPUT_RESOLUTION. The default value for this
parameter is 4 for all databases.
"poly/patches" is the total number of polygons and polygonal patches
generated when using OUTPUT_PATCHES.
"lights" is simply the number of lights in a scene. "background" is the
percentage of background color (empty space) seen directly by the eye for the
given view. "reflector" tells if there are reflective objects in the scene,
and "transmitter" if there are transparent objects.
"eye hit rays" is the number of rays from the eye which actually hit an
object (i.e. not the background). 513x513 eye rays are assumed to be shot
(i.e. one ray per pixel corner). "reflect rays" is the total number of rays
generated by reflection off of reflecting and transmitting surfaces. "refract
rays" is the number of rays generated by transmitting surfaces. "shadow rays"
is the sum total of rays shot towards the lights. Note that if a surface is
facing away from a light, or the background is hit, a light ray is not formed.
The numbers given can vary noticeably from a given ray tracer, but should all
be within about 10%.
"K" means exactly 1000 (not 1024), with number rounded to the nearest K.
All of the above statistics should be approximately the same for all
classical ray tracers.
Testing Procedures
------------------
Below are listed the requirements for testing various algorithms. These test
conditions should be realizable by most renderers, and are meant to represent
a common mode of operation for each algorithm. Special features which the
software supports (or standard features which it lacks) should be noted in
your statistics.
1) The non-polygon (OUTPUT_CURVES) format should normally be used for
ray-tracing tests.
2) All opaque (non-transmitting) primitives can be considered one-sided
for rendering purposes. Only the outside of primitives are visible in
the scenes. The only exception to this is the "teapot" database, in
which the teapot itself should normally be double sided (this is
necessary because the lid of the teapot does not fit tightly, allowing
the viewer to see back faces).
3) Polygonal patches (which are always triangles in the SPD) should have
smooth shading, if available.
4) Specular highlighting should be performed for surfaces with a
reflective component. The simple Phong distribution model is
sufficient.
5) Light sources are positional. If unavailable, assign the directional
lights a vector given by the light position and the viewing "lookat"
position.
6) Render at a resolution of 512 x 512 pixels, shooting rays at the
corners (meaning that 513 x 513 eye rays will be created). The four
corner contributions are averaged to arrive at a pixel value. If
rendering is done differently, note this fact. No pixel subdivision
is performed.
7) The maximum tree depth is 5 (where the eye ray is of depth 1). Beyond
this depth rays do not have to be spawned.
8) All rays hitting only reflective and transmitting objects spawn
reflection rays, unless the maximum ray depth was reached by the
spawning ray. No adaptive tree depth cutoff is allowed; that is, all
rays must be spawned (adaptive tree depth is a proven time saver and
is also dependent on the color model used - see Roy Hall's work for
details).
9) All rays hitting transmitting objects spawn refraction rays, unless
the maximum ray depth was reached or total internal reflection occurs.
Transmitting rays should be refracted using Snell's Law (i.e. should
not pass straight through an object). If total internal reflection
occurs, a reflection ray should still be generated at this node. Note
that all transmitters in the SPD are also reflective, but this is
not a requirement of the file format itself.
10) A shadow ray is not generated if the surface normal points away from
the light. This is true even on transmitting surfaces. Note any
changes from this condition.
11) Assume no hierarchy is given with the database (for example, color
change cannot be used to note a clustering). The ray tracing program
itself can create its own hierarchy, but this process should be
automatic. Note any exceptions to this (e.g. not including the
background polygon in the efficiency structure, changing the
background polygon into an infinite plane, etc). Such changes can be
critical for the efficiency of some schemes, so an explanation of why
changes were made is important.
12) Timing costs should be separated into at least two areas: preprocessing
and ray-tracing. Preprocessing includes all time spent initializing,
reading the database, and creating data structures needed to ray-trace.
Preprocessing should be all the constant cost operations--those that
do not change with the resolution of the image. Ray-tracing is the
time actually spent tracing the rays (i.e. everything that is not
preprocessing).
13) Other timing costs which would be of interest are in a breakdown of
times spent in preprocessing and during actual ray-tracing. Examples
include time spent reading in the data itself, creating a hierarchy,
octree, or item buffer, and times spent on intersection the various
primitives and on calculating the shade.
14) Time-independent statistics on the performance of the algorithm should
be gathered. Some examples are the number of ray/object intersection
tests, the number of ray/object tests which actually hit objects,
number of octree or grid nodes accessed, and the number of successful
shadow cache hits.
Timings
-------
Rendering time of the ray-traced test set on an HP-835, optimized (512 x 512):
Input Setup Ray-Tracing | Polygon Sphere Cyl/Cone Bounding
(hr:min:sec) | Tests Tests Tests Box Tests
-------------------------------------------------------------------------------
balls 0:14 0:19 24:56 | 822K 6197K 0 51726K
gears 0:56 0:18 1:02:31 | 13703K 0 0 107105K
mount 0:31 0:14 18:49 | 4076K 3978K 0 31106K
rings 0:41 0:33 53:41 | 1045K 5315K 16298K 91591K
teapot 1:02 0:18 25:38 | 7281K 0 0 57050K
tetra 0:15 0:06 3:48 | 965K 0 0 7637K
tree 0:40 0:31 11:36 | 479K 524K 1319K 22002K
Input: time spent reading in the NFF file.
Setup: time spent creating the database and any ray tracing efficiency
structures, and cleaning up after ray trace (does not include "Input" time).
Ray-Tracing: time spent traversing and rendering the pixel grid.
A typical set of ray tracing intersection statistics for the tetra database is:
[these statistics should be the same for all users]
image size: 512 x 512
total number of pixels: 262144 [ 512 x 512 ]
total number of trees generated: 263169 [ 513 x 513 ]
total number of tree rays generated: 263169 [ no rays spawned ]
number of eye rays which hit background: 213381 [ 81% - might vary ]
average number of rays per tree: 1.000000
average number of rays per pixel: 1.003910
total number of shadow rays generated: 46111 [ might vary a bit ]
[these tests vary depending on the ray-tracing algorithm used]
Intersector performance
Bounding box intersections: 7636497 - 26.71 usec/test
Polygon intersections: 964567 - 36.77 usec/test
Ray generation
Eye rays generated: 263169 ( 49788 hit - 18.92% )
Reflection rays generated: 0
Refraction rays generated: 0
Shadow rays generated: 46111
Coherency hits: 3407 - 7.39 % of total
Fully tested: 42704
The ray-tracer which generated these statistics is based on hierarchical
bounding boxes generated using Goldsmith & Salmon's automatic hierarchy method
(see IEEE CG&A May 1987). It no longer uses an item buffer, hence the higher
number of overall intersection tests from earlier SPD versions.
One problem worth analyzing when using the SPD for efficiency tests is how
octree, SEADS, and other space dividing algorithms perform when the background
polygon dimensions are changed (thus changing the size of the outermost
enclosing box, which changes the encoding of the environment). One analysis of
the effect of the background polygon on RayShade can be found in "Ray Tracing
News", volume 3, number 1.
Future Work
-----------
These databases are not meant to be the ultimate in standards, but are
presented as an attempt at providing representative modeled environments. A
number of extensions to the file format could be provided someday, along with
new database generators which use them. The present databases do not contain
polygons with holes, spline patches, polygonal mesh, triangular strip, or
polyhedron data structures. Modeling matrices are not output, and CSG
combined primitives are not included. Light sources are particularly
simplistic. For a richer, more user-friendly scene description language,
take a look at RayShade or RenderMan.
As far as database geometry is concerned, most scenes have a preponderance
of small primitives. If you find that these databases do not reflect the type
of environments you render, please write and explain why (or better yet, write
one or more programs that will generate your "typical" environments--maybe it
will get put in the next release).
Acknowledgements
----------------
I originally heard of the idea of standard scenes from Don Greenberg back
in 1984. Some time earlier he and Ed Catmull had talked over coming up with
some standard databases for testing algorithmic claims, and to them must go
the credit for the basic concept. The idea of making small programs which
generate the data came to me after Tim Kay generously sent huge files of his
tree database via email - I felt there had to be a better way. Adding the
teapot was inspired by the repeated demand on comp.graphics for this database,
certainly the most famous of all.
Many thanks to the reviewers, listed alphabetically: Kells Elmquist, Jeff
Goldsmith, Donald Greenberg, David Hook, Craig Kolb, Susan Spach, Rick Speer,
K.R. Subramanian, J. Eric Townsend, Mark VandeWettering, John Wallace, Greg
Ward, and Louise Watson. Other people who have kindly offered their ideas and
opinions on this project include Brian Barsky, Andrew Glassner, Roy Hall, Chip
Hatfield, Tim Kay, John Recker, Paul Strauss, and Chan Verbeck. These names
are mentioned mostly as a listing of people interested in this idea. They do
not necessarily agree (and in some cases strongly disagree) with the validity
of the concept or the choice of databases.
Your comments and suggestions on these databases are appreciated. Please
do send me any timing results for software and hardware which you test, or any
publications which use the SPD package.
