This is public beta release of Pavuk. Pavuk in Slovak language means spider. What this program does : -recursive HTTP , HTTP over SSL , FTP, FTP over SSL and Gopher document retrieving -supports HTTP/1.1 with persistent connections -supports HTTP POST requests -synchronizing retrieved local copies of document with remote -partial content retrieving on servers which suppots it (FTP and HTTP/1.1) -automaticaly follows moved documents -supports "robots.txt" standart -supports HTTP and FTP proxy servers -supports HTTP authentification (user, Basic, Digest access) -shows document tree -have interface to "at" command for scheduling -have optional GTK+ user interface -may be built with or without X Window user interface -can handle setup files -supports NLS via GNU gettext (so messages are translatable to many native languages) -can be used for fetching documents to proxy/cache server (-mode dontstore) -supports HTTP cookies -supports SOCKS(4/5) proxy -you can run as many as instances of pavuk in same tree without any loose of data, because pavuk locks documents while procesing -you can limit transfer rate over network -have powerfull mechanism for mapping URLs to local filenames -can check URL if it is modified and can send list of modified URLs to any script -can load files from Netscape browser cache directory -can filter advertisement banners -can automaticaly turn on/off output of messages to terminal when running in foreground or background -can run users postprocessing scripts for each downloaded document -optionaly can generate statistical reports from download, usable for WEB site debuging -supports different FTP directory listing formats (SYSV, BSD, EPFL, NOVEL, VMS, DOS/WINDOWS) -multithreading support ....? License : Everything is licensed under GPL. See COPYING and COPYING.LIB. Ported to: -Linux (x86) (gcc) -Digital Unix 3.2 (cc and gcc),4.0 (cc and egcs) -Ultrix 4.4 (cc,egcs) -NetBSD (sparc , mips) (gcc) /* last versions not tested */ -GNU win32 (gcc + cygwin) /* only cli interface yet */ -Solaris 7 (x86) (gcc) -also several other UNIX-es, Mac OS X, OS/2 reported to work Author : Ondrejicka Stefan (ondrej@idata.sk) Home page: http://www.idata.sk/~ondrej/pavuk/ If you find any bugs please send me report and or fix. If you have any suggestions, ideas or quetions please write me.
