Comments on: Plotting

By: Austringer

Austringer — Tue, 05 Feb 2008 14:16:22 +0000

Zarquon: Yes, FreeBSD does have gnuplot. For some reason, the gnuplot package didn’t install properly for 6.3-release. I have since installed from ports, but I got “gri” via pkg_add quicker.

Ben M: Thanks, yes, it pays to take care. I also took the correlation test from Zar and wrote a Perl script to implement it via various machine formulae for the terms. The correlation is both reasonably strong and significant. I wouldn’t rely upon a graph for more than a visual confirmation of what the statistics tell.

Greg Laden: My PI is not in favor of pre-publishing data or results, so I’ll have to defer putting the plot up for a bit. I did like the demo graphs you had for “gri” in your post, and I think those show capabilities nicely.

By: Ben M

Ben M — Mon, 04 Feb 2008 23:29:32 +0000

You have to be careful with that many points. Chances are, you’re *looking* at those 5M points spread out over, say, 800×800 pixels = 0.64M. There *will* be overlapping points. You say you can see the correlation—well, it’s possible that the correlation you see is just the outliers, the ones that don’t overlap.

For example, imagine that you’re plotting points between 0

By: Zarquon

Zarquon — Sun, 03 Feb 2008 09:12:54 +0000

FreeBSD ports doesn't have gnuplot?

By: Greg Laden

Greg Laden — Sat, 02 Feb 2008 17:08:14 +0000

Nice, thanks for that tip.

On why plot the points? Well, I agree with Austringer. If you use a method where you plot the points if there are 50, 0r 500, etc., then it is appropriate to plot them out even if there are 5 million.

You must promise us this, though: Let’s see the plot!!!

By: George

George — Sat, 02 Feb 2008 16:37:43 +0000

Still seems like you’re going for thud factor. Statistics might be your best friend. I would suspect that a graph made from a sample would properly make your point.

I would love to see the graph – any chance you can post it for show and tell?

By: Austringer

Austringer — Thu, 31 Jan 2008 06:10:52 +0000

Actually, I’m finding that my install of StatView, a Windows 3.1 program originally, handles my dataset okay under Windows Vista. It is slow, but there hasn’t been any tendency to crash.

By: Frank J

Frank J — Wed, 30 Jan 2008 14:33:31 +0000

(sigh)

Now I know how nonscientists feel when a defender of science takes the bait and responds to a creationist sound bite with a technical refutation.

If you weren’t on my short list of heroes defending science education, I’d swear that you were part of the Evil Computer Geek branch of the nonexistent EAC.

If you want a laugh, I use Excel to plot XY graphs, and yes, I did overload it a few times.

By: Austringer

Austringer — Tue, 29 Jan 2008 21:08:09 +0000

The basic aim is to show the correlation between the x and y variables. This does come through well enough with a scatterplot. 5.2M is the total population size.

By: George

George — Tue, 29 Jan 2008 20:38:08 +0000

As Bob O’H: Exactly, why so many data points on a plot? I am facinated by what the need could be?

By: Bob O'H

Bob O'H — Tue, 29 Jan 2008 17:12:32 +0000

Why would you want to plot 5.2 million data points? You’ll just have a black smudge. Or a very large graph.

Why not plot the density? I use some R code which calculates and plots contours. If you want the code, email me.

Or you could bin the data (you have enough!), and plot those as contours, or as 3D plots.

Bob