GraphDefang FAQ

1. What is the difference between libgd, GD.pm, and GD::Graph? Which ones do I need to use graphdefang?
You need all of them!

The libgd library is an ansi C library that dynamically produces png and jpeg files. libgd is written by Thomas Boutell. It can be found at http://www.boutell.com/gd/. The older 1.8.4 version comes with Redhat 7.2, 7.3, and 8.0 with the rpm name gd. I have successfully used 1.8.4 and the newer 2.0.X series on Redhat 7.2, Redhat 7.3, and NetBSD 1.4.

The GD perl library is also called GD.pm, and it is a perl module that allows you to use the libgd ANSI C library calls from perl. The GD perl module is written by Lincoln Stein. It can be found at http://stein.cshl.org/WWW/software/GD/ Version 1.40 of this module works with the libgd 1.8.4 that comes with RedHat.

The GD::Graph library is a perl module that sets up typical graph functions in the GD graphics library. It builds on and uses the GD perl library to access the ANSI C libgd libraries. You can get the latest version from CPAN. It is written by Martien Verbruggen.

2. Just wondering.. maybe you know about this. I'd love to use your graphdefang program..but i'm having a hell of a time installing the perl GD module. throws all kinds of errors.. stuff like this

GD.xs:1059: invalid lvalue in assignment
GD.xs: In function `XS_GD__Image_rotate180':
GD.xs:1079: invalid lvalue in assignment
GD.xs:1079: invalid lvalue in assignment
GD.xs:1080: invalid lvalue in assignment
GD.xs:1080: invalid lvalue in assignment


You need to make sure you are using compatible versions of libgd and GD.pm. Specifically, RedHat ships with version 1.8.4 (try 'rpm -q gd' to determine your version.), and it is not compatible with newer versions of GD.pm.

Here's the bottom line... if you have libgd 1.8.4, use GD.pm version 1.40.

If you wish, you may upgrade your libgd to the new version, but pay attention to the installation path. The 2.0.7 version of libgd installs by default to /usr/local/lib. When you are creating the Makefile for GD.pm, it will ask you the path to your libgd files... make sure you enter the path to the libraries that you just installed!

3. I can run graphdefang successfully once, but the second time I run it, it spits out the following message for most of the graphs:

 Processing graphs
          hourly_all_summary_line
 gd-png:  fatal libpng error: Invalid number of colors in palette
 gd-png error: setjmp returns error condition
          daily_all_summary_line
 gd-png:  fatal libpng error: Invalid number of colors in palette
 gd-png error: setjmp returns error condition
          monthly_all_summary_line
 gd-png:  fatal libpng error: Invalid number of colors in palette
 gd-png error: setjmp returns error condition
          hourly_virus_value1_stacked_bar
 gd-png:  fatal libpng error: Invalid number of colors in palette
 gd-png error: setjmp returns error condition

I've seen this on 2 different BSD based systems. When it was all said and done, the problem was with the Berkeley DB installation on both systems. After spending quite a bit of time troubleshooting the problem, I finally came up with the following solution:

Okay... I freaking got it!

Here's what I had to do:

edit:  the versions mentioned below don't have any special magic.  They 
were just the versions that were current when I went through and did this
the first time.  Go ahead and use whatever is current on the sleepycat site.

1.  Download BerkeleyDB v4.1.24 from
http://www.sleepycat.com/update/snapshot/db-4.1.24.NC.tar.gz

2.  untar file, and run make inside of the dist/ directory.

3.  run make install inside of the dist/ directory

4.  it installed everything in /usr/local/BerkeleyDB.4.1

5.  symbolically link the BerkeleyDB.4.1 directory to BerkeleyDB such that
/usr/local/BerkeleyDB points to the directory that was created by make
install.

6.  goto the perl/DB_File directory of the db-4.1.24 distribution.  run
perl Makefile.PL, make, make test, make install

Then... it worked!  The symbolic link was necessary for DB_File to find
the new libraries to link against.  You may have to do this or something
else.  Just make sure it is linking against the new libraries...

Try it and let me know if it worked for you!

4. I just installed GraphDefang. I am seeing some data on the graphs, but the Mail_in value is always 0. What can I check?
Old Answer:

I have a line in my mimedefang-filter file that adds that event for all
non-spam email that comes in to my mail server.

I have the following after the filter determines that the message is not
spam:

if (defined($SendmailMacros{'auth_author'})) {
    md_graphdefang_log('mail_out',,$RelayAddr);
} else {
    md_graphdefang_log('mail_in',,$RelayAddr);
}

I don't even remember what the sendmail macro, 'auth_author', means, but
in a series of tests that I ran, it was set in most outgoing mail and not
set in most incoming mail.  There were some crossover cases, but they were
a significant minority.

New Answer:

Now... I use a different method.  The relay address, $RelayAddr, is being 
compared to localhost (127.0.0.1) and the IP number of the machine 
(192.168.0.129).  If the relay address matches either of these, then I'm 
calling it 'mail_out'.  Any other relay address, and it is spam or mail_in 
(or probable spam -- which I also have in my filter).  ALSO... this example
doesn't check for spam in outgoing email -- that's how I have mine setup.

Pseudo-code is below:

	# Spam checks if SpamAssassin is installed
	if ($Features{"SpamAssassin"}) {
		if ($RelayAddr eq "127.0.0.1" or
		    $RelayAddr eq "192.168.0.129") {

			md_graphdefang_log('mail_out',,$RelayAddr);

		} else {
			# Check for spam
			# stuff cut out

			if ($spam) {
				md_graphdefang_log('spam', $hits, $RelayAddr);
				# more stuff cut out
			} else {
				md_graphdefang_log('mail_in',,$RelayAddr);
			}
		}
	}

5. I use a BSD (Free, Net, Open, etc.) or Solaris, and I can't seem to get the summary database to function correctly.

I have had this reported from a large number of users, and the solution in all
cases (so far) has been to follow the same steps as for FAQ #3.  For a reason 
that is currently un-investigated by me, the default install of Berkeley DB 
and the Perl DB_File just don't seem to be right on these OSes.  Even installing
things from pkgsrc or whatever packaging system is available didn't fix the issue
for some people -- they had to go back and do it manually.

6. I lost all of my historical data! What could have caused this?

This has been reported to me about 5 times in non-really-high-volume-mail
installations.  In 4 of the cases, the user had done something (upgrade 
perl, usually) that undid what they did in FAQ #3.  In the last case, 
I have no idea... something ate it!

There is also something that likely happens with large SummaryDB files.  
I don't know how to quantify large here.  It works great for me with a 
couple thousand messages per day on an incoming mail server, and it 
works with a couple of tens of thousands of messages per day on an 
outgoing mail server.  People have reported core dumps on processing
a volume of millions of messages a day.  I don't know what causes this.

7. The performance is quite poor on large datasets! What might I do to help fix this?

Yes, you are correct.  The performance can be poor on large datasets.  You should
trim your SummaryDB using the "trim" argument to graphdefang.  Also, there was a
defect in all releases of graphdefang prior to 0.9 that caused the "trim" argument
to not do all it was capable of.  After fixing the defect, my SummaryDB file shrunk
by 10 times!  This increased my performance significantly.