Monthly ArchiveOctober 2012

Law and Politics &Media Wesley R. Elsberry on 23 Oct 2012

## False Balance in Fact Check

The Chicago Tribune Fact Check on the debates goes for that false balance thing in the headline:

“FACT CHECK: Romney flubs geography, Obama goofs on rival’s record, in final debate”

Mitt Romney, candidate for Commander-in-Chief, who not long ago identified Iran as a tippy-top threat to the security of the USA, has no clue about Iran’s geography or military disposition, which includes lots of threats to close the Strait of Hormuz to shipping. President Obama, on the other hand, is unlikely to launch a war on Massachusetts over the historical footnote that is Mitt Romney’s single-term governorship. Would it have been better if Obama knew more about the details of Mitt’s gubernatorial history? Sure. Is it anywhere comparable to Mitt’s thorough-going ignorance of current world affairs? I don’t think so.

<> 83949 5508 >

Computation &Science &Wildlife Wesley R. Elsberry on 08 Oct 2012

## Population Modeling in Python

One of the courses I enjoyed most in my Ph.D. program was taught by Prof. Kirk Winemiller on population dynamics. There are various collections of models in various languages out there, and multi-model population dynamic applications. But I still think that there is some utility to rolling my own. Since 2009, I’ve gotten more into Python programming, so I thought that I would take a popular class of population dynamic models and produce a Python module to instantiate them.

A long-time standard method in population modeling is the Leslie matrix. This technique applies when one has data about the age structure of a population and produces estimates going forward by using matrix multiplication to go from the population numbers, fecundity, and survivorship numbers to get the estimate of the population in each age class at the next time step.

A similar method is the Lefkovitch approach. This is still based upon matrix operations, but the underlying data involves stages rather than age structure. This sort of model is often used to capture more complex life histories than are tracked in a Leslie matrix model.

The similarities make it straightforward to incorporate both approaches into one supporting Python class.

The following Python module defines the LMatrix class. The dependencies are the Numpy module and the interval module. I used “pip install interval” to get the interval module on my machine. If you run this module in standalone mode, it runs a test of the LMatrix model with a web-accessible example of a Leslie matrix and of a Lefkovitch matrix.

1. """
2. popdyn.py
3.
4. Trying out population dynamics in Python.
5. Wesley R. Elsberry
6.
7. """
8.
9. class LMatrix:
10.     """
11.    LMatrix
12.
13.    A support class for Leslie and Lefkovitch matrix use for
14.    population dynamics.
15.
16.    This is a generic class that allows for an arbitrary number of age
17.    classes or stages.
18.    """
19.
20.     def __init__(self,stAges):
21.         import numpy as num
22.         import numpy.matlib as M
23.         from numpy.matlib import rand,zeros,ones,empty,eye
24.         import interval
25.
26.         """
27.        In either Leslie age-structured or Lefkovitch stage-
28.        structured population modeling, the central feature
29.        is a special matrix representing both fecundity of
30.        ages/stages and survivorship in each age/stage.
31.
32.        The Leslie age-structured matrix is slightly simpler,
33.        since each iteration moves the population forward
34.        by a time step equal to the difference between the
35.        age classes.
36.
37.        The Lefkovitch stage-structured matrix,
38.        on the other hand, may have unequal times spent in
39.        each stage, and thus other elements of the matrix
40.        represent the fraction of individuals that continue
41.        to remain in the stage per time step of the model.
42.        Those lie on the main diagonal.
43.
44.        The matrix in either case is an N-by-N matrix, where
45.        N is the number of ages or stages (stAges parameter).
46.        Because most values in the matrix are zero, we'll
48.        """
49.
50.         self.stAges = stAges  # Keep track of how many age/stage classes there are
51.         self.m = zeros((self.stAges,self.stAges))
52.         self.step = 0  # We are at the beginning
53.         self.popvec = None
54.         self.survival = None
55.         self.recurrence = None
56.         self.fecundity = None
57.
59.         """
60.        Method to set fecundity values for an LMatrix.
61.
62.        This is done by setting the first row of the
63.        matrix to the values in the vector.
64.
65.        A mismatch between the length of the vector and
66.        the width of the matrix leaves both unchanged.
67.        """
68.         if (fvector.shape[0] == self.stAges):
69.             # Just replace the row
70.             self.m[0] = fvector
71.             # Save it in the object
72.             self.fecundity = fvector
73.         else:
74.             print "Mismatch in size: %s vs. %s" % (self.stAges - 1,fvector.shape[0])
75.
77.         """
78.        Add the values for survival that shift population members
79.        from one age/stage to the next.
80.        The values come in as the "survival" vector, a Numpy array.
81.        They replace values in the m matrix in the diagonal from
82.        [1,0] to [N-1,N-2].
83.        """
84.         if (survival.shape[0] == (self.stAges - 1)):
85.             for ii in range(1,self.stAges):
86.                 self.m[ii,ii-1] = survival[ii-1]
87.             # Save it in the object
88.             self.survival = survival
89.         else:
90.             print "Mismatch in size: %s vs. %s" % (self.stAges - 1,survival.shape[0])
91.
93.         """
94.        Add the values for survival of organisms remaining in the same
95.        stage. This is for stage-structured population models only.
96.        The input is as the vector recur, and its values replace those
97.        in the m matrix along the main diagonal from [1,1] to [N-1,N-1].
98.        """
99.         if (recur.shape[0] == (self.stAges - 1)):
100.             for ii in range(1,self.stAges):
101.                 self.m[ii,ii] = recur[ii-1]
102.             # Save it in the object
103.             self.recurrence = recur
104.         else:
105.             print "Mismatch in size: %s vs. %s" % (self.stAges - 1,recur.shape[0])
106.
107.     def LM_SetOneRelation(self,fromState,toState, value):
108.         """
109.        Method to set a relation that does not fall on the survival
110.        diagonal or the recurrence diagonal. This is useful for more
111.        complex stage-structured population modeling where organisms
112.        from one stage may graduate to multiple other stages at defined
113.        rates.
114.        """
115.         iv = interval.Interval.between(0,self.stAges-1)
116.         if ((fromState in iv) and (toState in iv)):
117.             print self.m
118.             self.m[toState,fromState] = value
119.             print self.m
120.
121.     def LM_SetPopulation(self,popvector):
122.         """
123.        Another central feature of these models is that the size
124.        of the population is kept in a 1xN column vector. For the
125.        implementation here, the actual representation is as a
126.        Numpy array, which has no column vector as such. This will
127.        be handled in the actual stepping method.
128.        """
129.         if (popvector.shape[0] == (self.stAges)):
130.             self.popvec = popvector
131.         else:
132.             print "Mismatch in size: %s vs. %s" % (self.stAges,popvector.shape[0])
133.
134.     def LM_StepForward(self):
135.         """
136.        Do the matrix multiplication to obtain the new population
137.        vector. Retain the previous population vector.
138.
139.        Handle turning population vector into a column vector for the
140.        multiplication.
141.        """
142.         # Convert the population array to a Numpy matrix and transpose it
143.         # to get the column vector we need. Multiply the L* matrix by
144.         # the column vector, resulting in a new column vector with the
145.         # population at the next step.
146.         nextpopvec = num.mat(self.m) * num.mat(self.popvec).T
147.
148.         # Save the old population vector
149.         self.lastpopvec = self.popvec
150.
151.         # Replace the population vector with the new one, which means
152.         # transposing it and converting to Numpy array type
153.         self.popvec = num.array(nextpopvec.T)
154.
155.         # Track the number of steps taken
156.         self.step += 1
157.
158.     def LM_TotalPopulation(self):
159.         """
160.        Return the total population size. Sums the "popvec" vector.
161.        """
162.         if (None != self.popvec):
163.
164.             # Population vector as array multiplied by column vector of 1s is a sum
165.             t = num.mat(self.popvec) * ones(self.stAges).T
166.
167.             return t[0,0]
168.         else:
169.             return 0.0
170.
171.
172. if __name__ == "__main__":
173.     """
174.     Generic initialization suggested at
175.     http://www.scipy.org/NumPy_for_Matlab_Users
176.    """
177.     # Make all numpy available via shorter 'num' prefix
178.     import numpy as num
179.     # Make all matlib functions accessible at the top level via M.func()
180.     import numpy.matlib as M
181.     # Make some matlib functions accessible directly at the top level via, e.g. rand(3,3)
182.     from numpy.matlib import rand,zeros,ones,empty,eye
183.     # Define a Hermitian function
184.     def hermitian(A, **kwargs):
185.         return num.transpose(A,**kwargs).conj()
186.
187.     # Make some shorcuts for transpose,hermitian:
188.     #    num.transpose(A) --> T(A)
189.     #    hermitian(A) --> H(A)
190.     T = num.transpose
191.     H = hermitian
192.
193.     import interval
194.
195.
196.     # Check it against an existing example data set
197.     # http://www.cnr.uidaho.edu/wlf448/Leslie1.htm
198.
199.     ex1 = LMatrix(4)
200.
201.     fex1 = num.array([0.5, 2.4, 1.0, 0.0])
203.
204.     sex1 = num.array([0.5, 0.8, 0.5])
206.
207.     pex1 = num.array([20, 10, 40, 30])
208.     ex1.LM_SetPopulation(pex1)
209.
210.     print pex1
211.     print ex1.m
212.     ex1.LM_StepForward()
213.     print ex1.popvec
214.
215.     # It checks out!
216.
217.     # Another example, this time of a stage-structured population
218.     # http://www.afrc.uamont.edu/whited/Population%20projection%20models.pdf
219.     ex2 = LMatrix(3)
220.
221.     fex2 = num.array([0.0, 52, 279.5])
223.
224.     sex2 = num.array([0.024, 0.08])
226.
227.     rex2 = num.array([0.25, 0.43])
229.
230.     pex2 = num.array([70.0,20.0,10.0])
231.     ex2.LM_SetPopulation(pex2)
232.
233.     print pex2
234.     print ex2.m
235.     ex2.LM_StepForward()
236.     print ex2.popvec
237.     print ex2.LM_TotalPopulation()
238.
239.     ex2.LM_StepForward()
240.     print ex2.LM_TotalPopulation()
241.
242.     ex2.LM_StepForward()
243.     print ex2.LM_TotalPopulation()
244.
245.     for ii in range(22):
246.         ex2.LM_StepForward()
247.     print ex2.popvec
248.
249.     # Tests OK!

Output from the standalone run:

1. [20 10 40 30]
2. [[ 0.5  2.4  1.   0. ]
3.  [ 0.5  0.   0.   0. ]
4.  [ 0.   0.8  0.   0. ]
5.  [ 0.   0.   0.5  0. ]]
6. [[ 74.  10.   8.  20.]]
7. [ 70.  20.  10.]
8. [[  0.00000000e+00   5.20000000e+01   2.79500000e+02]
9.  [  2.40000000e-02   2.50000000e-01   0.00000000e+00]
10.  [  0.00000000e+00   8.00000000e-02   4.30000000e-01]]
11. [[ 3835.       6.68     5.9 ]]
12. 3847.58
13. 2093.1914
14. 5811.535142
15. [[ 19837904.89838918    393232.36554185     30519.85368983]]
<> 86245 6032 >

General Wesley R. Elsberry on 07 Oct 2012

## Anatomy of a Spam Comment

The new spam module for WordPress is not as aggressive in marking spam messages as its previous version. So I get to go through and mark a variety of things that are pretty obviously spam as spam.

Well, today a spammer’s script misfired and sent his template for a comment rather than a processed spam comment itself. For the morbidly curious, I’ll quote it here:

{Pretty|Attractive} section of content. I just stumbled upon your {blog|weblog|website|web site|site} and in accession capital to assert that I {acquire|get} {in fact|actually} enjoyed account your blog posts. {Any way|Anyway} {I?ll|I will} be subscribing to your {augment|feeds} and even I achievement you access consistently {rapidly|fast|quickly}.

The obvious way this kind of thing works is that the script gets a list of websites to crawl and look for comment blocks. Then, it should use the template to generate a plaintext message using only on alternative from each of the selections in curly braces. This helps keep the spam detection software off guard, since with even a few alternatives at each of several positions, the permutations can reach an astounding number. In this case, whatever script was supposed to actually select alternatives and emit plaintext obviously failed. Given how broken the grammar is, perhaps one should expect the coding to be of similar quality.

<> 83287 5422 >

CafePress Shop