<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" media="screen" href="/~d/styles/rss2full.xsl"?><?xml-stylesheet type="text/css" media="screen" href="http://feeds.feedburner.com/~d/styles/itemcontent.css"?><rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0" version="2.0">

<channel>
	<title>Ed's Big Plans</title>
	
	<link>http://eddiema.ca</link>
	<description>Computing for Science and Awesome</description>
	<lastBuildDate>Thu, 02 Feb 2012 23:00:43 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="self" type="application/rss+xml" href="http://feeds.feedburner.com/eddiema" /><feedburner:info uri="eddiema" /><atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="hub" href="http://pubsubhubbub.appspot.com/" /><item>
		<title>Fighting spam account registration (phpBB3)</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/FPlLTILtGjs/</link>
		<comments>http://eddiema.ca/2011/12/29/fighting-spam-account-registration-phpbb3/#comments</comments>
		<pubDate>Thu, 29 Dec 2011 21:49:21 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Website Management]]></category>
		<category><![CDATA[bot]]></category>
		<category><![CDATA[fake account]]></category>
		<category><![CDATA[linkedin]]></category>
		<category><![CDATA[phpBB]]></category>
		<category><![CDATA[phpBB3]]></category>
		<category><![CDATA[registration]]></category>
		<category><![CDATA[spam]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=3980</guid>
		<description><![CDATA[I was asked to deploy a means to fight spam account registration on a phpBB3 installation. I tried the advice posted by Nadav Samet, but the version of phpBB3 we use doesn&#8217;t seem to respond to changes made to profile_add_body.tpl. I settled on an approach that&#8217;s similar to the one posted by Steve Maury, but is slightly [...]]]></description>
			<content:encoded><![CDATA[<p>I was asked to deploy a means to fight spam account registration on a phpBB3 installation. I tried the advice posted by <a href="http://www.thesamet.com/blog/2006/12/21/fighting-spam-on-phpbb-forums/">Nadav Samet</a>, but the version of phpBB3 we use doesn&#8217;t seem to respond to changes made to <em>profile_add_body.tpl</em>. I settled on an approach that&#8217;s similar to the one posted by <a href="http://www.phpbb.com/kb/article/custom-profile-fields-as-an-anti-spammer-tool/">Steve Maury</a>, but is slightly improved as you can request any exact string for an answer from the registrant.</p>
<p>This method should work with phpBB3 version 3.0.x.</p>
<p>This method results in a mandatory field to be completed by the registrant during registration time. You can assign it any question you like where the answer is a string of text that you decide. I&#8217;ve given a <span style="text-decoration: underline;"><strong>really simple question in this example</strong></span>. Use a more difficult question that a bot would have trouble with but that your typical registrants can answer.</p>
<p>We do this with a custom profile field in the registration screen as in the below screen capture.</p>
<p style="text-align: center;"><img class="size-full wp-image-4322 aligncenter" title="GryphQuestion" src="http://eddiema.ca/wp-content/uploads/2011/12/GryphQuestion1.png" alt="" width="443" height="400" />The above is a screen capture of an example mandatory question (choose a more secure question).</p>
<p><strong>There are two steps to setting this up.</strong></p>
<p><span style="text-decoration: underline;"><strong>First,</strong> set up a custom question that must be answered</span> &#8212; you can find such a setting in the administrative control panel (ACP) as in the following screen capture.</p>
<p style="text-align: center;"><img class="alignnone size-full wp-image-4340" title="CustomProfileFields" src="http://eddiema.ca/wp-content/uploads/2011/12/CustomProfileFields1.png" alt="" width="694" height="330" /><br />
Add a mandatory question in <em>user and groups</em> → <em>custom profile fields</em> → <em>create new field</em>.<br />
Here, I&#8217;ve named mine <em>guelph_question</em>.</p>
<p style="text-align: left;">After you click <em>Create new field</em>, you will be presented with a screen where you can create the question of your choosing &#8212; be sure to make the field mandatory by using the radio button labelled <em>Required field</em> listed under <em>Visibility options</em>.</p>
<p style="text-align: left;"><span style="text-decoration: underline;"><strong>Second,</strong> add the following code indicated below</span> in <span style="font-family: 'andale mono', times;">includes/ucp/ucp_register.php</span> &#8211; Notice that I&#8217;ve placed my custom code after existing code to check for a valid password and e-mail. This is commented under <em>validate custom profile fields </em>( &#8212; just checked &#8212; starts at line 247 in the 3.0.9 source code). Replace the simple question and answer I&#8217;ve placed there with your own question and answer. Notice that you also have full control over the error message that the registrant sees if they answer the question incorrectly.</p>
<pre class="brush: php; title: ; notranslate">
// validate custom profile fields
$cp-&gt;submit_cp_field('register', $user-&gt;get_iso_lang_id(), $cp_data, $error);

if (!sizeof($error))
{
	if ($data['new_password'] != $data['password_confirm'])
	{
		$error[] = $user-&gt;lang['NEW_PASSWORD_ERROR'];
	}

	if ($data['email'] != $data['email_confirm'])
	{
		$error[] = $user-&gt;lang['NEW_EMAIL_ERROR'];
	}
}

// Anti-spam code below ...
if (!sizeof($error)) {
	if (!isset($_POST['pf_guelph_question']) ||
	$_POST['pf_guelph_question'] != 'Gryphon') {
		$error[] = 'Incorrect answer to Guelph question.';
	}
}
// ... EOAnti-spam code
</pre>
<p>And you&#8217;re done.</p>
<p>I hope this works for you! Enjoy <img src='http://eddiema.ca/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/FPlLTILtGjs" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/12/29/fighting-spam-account-registration-phpbb3/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/12/29/fighting-spam-account-registration-phpbb3/</feedburner:origLink></item>
		<item>
		<title>Fuzzy c-means for greylevel image segmentation</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/5Sf_wF-bfOw/</link>
		<comments>http://eddiema.ca/2011/12/20/fuzzy-c-means-for-greylevel-image-segmentation/#comments</comments>
		<pubDate>Wed, 21 Dec 2011 04:21:54 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Algorithms]]></category>
		<category><![CDATA[Featured]]></category>
		<category><![CDATA[Auryo]]></category>
		<category><![CDATA[FCM]]></category>
		<category><![CDATA[Fuzzy C-Means]]></category>
		<category><![CDATA[Greylevel Segmentation]]></category>
		<category><![CDATA[linkedin]]></category>
		<category><![CDATA[Oreo]]></category>
		<category><![CDATA[Sunny]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=3482</guid>
		<description><![CDATA[Here&#8217;s a script I threw together to do grey-level segmentation using fuzzy c-means. This appeared as a small part of a project in the image processing course I took. The algorithm deployed was really a proof of concept meant to replicate and verify the results of another author &#8212; as such, I don&#8217;t recommend ever [...]]]></description>
			<content:encoded><![CDATA[<p>Here&#8217;s a script I threw together to do grey-level segmentation using fuzzy c-means. This appeared as a small part of a project in the image processing course I took. The algorithm deployed was really a proof of concept meant to replicate and verify the results of another author &#8212; as such, I don&#8217;t recommend ever using fuzzy c-means for this task as it&#8217;s pretty inefficient. This software will handle any number of grey-level-segments you desire, but I recommend eight as a maximum.</p>
<p><em>The code and course project paper are originally dated April 20th, 2011.</em></p>
<p><span style="font-family: helvetica;">&gt;&gt;&gt; <strong>Download:</strong> <a href="http://eddiema.ca/wp-content/uploads/2011/12/FCMProjectPaper.pdf">FCMProjectPaper.pdf</a> | <a href="http://eddiema.ca/wp-content/uploads/2011/12/FCMProject.tgz">FCMProject.tgz</a> &lt;&lt;&lt;</span></p>
<p>Here are the examples included in the above archive. I like pictures.</p>
<div style="text-align: center;"><img class="alignnone  wp-image-4278" style="border-width: 0px; margin: 0px;" title="sunny2" src="http://eddiema.ca/wp-content/uploads/2011/12/sunny2-300x239.png" alt="" width="216" height="172" /><img class="alignnone  wp-image-4277" style="border-width: 0px; margin: 0px;" title="sunny2_segmented8" src="http://eddiema.ca/wp-content/uploads/2011/12/sunny2_segmented8-300x239.png" alt="" width="216" height="172" /><img class="alignnone  wp-image-4276" style="border-width: 0px; margin: 0px;" title="sunny2_segmented4" src="http://eddiema.ca/wp-content/uploads/2011/12/sunny2_segmented4-300x239.png" alt="" width="216" height="172" /></div>
<p>Sunny in 8-bit greyscale, 3-bit greyscale, 2-bit greyscale.</p>
<div style="text-align: center;"><img class="alignnone  wp-image-4275" style="border-width: 0px; margin: 0px;" title="oreo1" src="http://eddiema.ca/wp-content/uploads/2011/12/oreo1-300x239.png" alt="" width="216" height="172" /><img class="alignnone  wp-image-4274" style="border-width: 0px; margin: 0px;" title="oreo1_segmented8" src="http://eddiema.ca/wp-content/uploads/2011/12/oreo1_segmented8-300x239.png" alt="" width="216" height="172" /><img class="alignnone  wp-image-4273" style="border-width: 0px; margin: 0px;" title="oreo1_segmented4" src="http://eddiema.ca/wp-content/uploads/2011/12/oreo1_segmented4-300x239.png" alt="" width="216" height="172" /></div>
<p>Auryo in 8-bit greyscale, 3-bit greyscale, 2-bit greyscale.</p>
<p>Enjoy <img src='http://eddiema.ca/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/5Sf_wF-bfOw" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/12/20/fuzzy-c-means-for-greylevel-image-segmentation/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/12/20/fuzzy-c-means-for-greylevel-image-segmentation/</feedburner:origLink></item>
		<item>
		<title>My talk at Barcode of Life, Adelaide (2011)</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/iASqdxMcILs/</link>
		<comments>http://eddiema.ca/2011/12/03/my-talk-at-barcode-of-life-adelaide-2011/#comments</comments>
		<pubDate>Sat, 03 Dec 2011 22:12:05 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Computational Biology]]></category>
		<category><![CDATA[Featured]]></category>
		<category><![CDATA[Adelaide]]></category>
		<category><![CDATA[Automation]]></category>
		<category><![CDATA[Barcode of Life]]></category>
		<category><![CDATA[BOLD]]></category>
		<category><![CDATA[Contig Assembly]]></category>
		<category><![CDATA[linkedin]]></category>
		<category><![CDATA[Sujeevan Ratnasingham]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=4212</guid>
		<description><![CDATA[I&#8217;ve just finished my presentation in Adelaide. This is the first real biology-heavy conference I&#8217;ve been to. Sujeevan has brought me along with the BOLD team from BIO in order to present my work &#8212; and more importantly &#8212; to acquire some resolution about the barcoding culture and its biological significance. The Consortium for The [...]]]></description>
			<content:encoded><![CDATA[<p>I&#8217;ve just finished my presentation in Adelaide. This is the first real biology-heavy conference I&#8217;ve been to. Sujeevan has brought me along with the <a href="http://boldsystems.org/">BOLD</a> team from <a href="http://www.biodiversity.uoguelph.ca/">BIO</a> in order to present my work &#8212; and more importantly &#8212; to acquire some resolution about the barcoding culture and its biological significance. The <a href="http://www.barcodeoflife.org/">Consortium for The Barcode of Life</a> (CBOL) co-hosted this event with many biodiversity parties in Australia. Another huge group present was the <a href="http://ibol.org/">International Barcode of Life</a> (iBOL) project. The Barcode of Life Conference is held every other year and is attended by researchers interested in the concerted international barcoding effort. I presented my preliminary findings with a data analysis session and had excellent feedback &#8212; it&#8217;s pretty clear where to go next with my thesis! My talk describes the first steps to automating barcode (contig-like) assembly from ab1 sequencer trace files. This talk describes the present need for automation, trends that we can readily detect in currently assembled data and most importantly &#8212; detectable patterns in how human experts perform manual barcode assembly.</p>
<p>The full name of the conference is <em><a href="http://www.dnabarcodes2011.org/">Fourth International Barcode of Life Conference</a></em>.</p>
<p><span style="font-family: tahoma, arial, helvetica, sans-serif;">&gt;&gt;&gt; <strong>Download:</strong> ( pdf: <a href="http://eddiema.ca/wp-content/uploads/2011/12/EddieMaBOL2011Adelaide.pdf">EddieMaBOL2011Adelaide.pdf</a> | zip: <a href="http://eddiema.ca/wp-content/uploads/2011/12/EddieMaBOL2011Adelaide.pdf.zip">EddieMaBOL2011Adelaide.pdf.zip</a> ) &lt;&lt;&lt;</span></p>
<div style="text-align: center;"><img class="size-full wp-image-4215" title="adelaide_06" src="http://eddiema.ca/wp-content/uploads/2011/12/adelaide_06.png" alt="" width="230" /><img class="size-full wp-image-4216" title="adelaide_16" src="http://eddiema.ca/wp-content/uploads/2011/12/adelaide_16.png" alt="" width="230" /><img class="size-full wp-image-4217" title="adelaide_24" src="http://eddiema.ca/wp-content/uploads/2011/12/adelaide_24.png" alt="" width="230" /></div>
<p style="text-align: center;">Slides 6, 16, 24 from my presentation &#8212; The need for automation; Compositional bias and human edits [null hypothesis]; Where are human edits occurring [in Lepidoptera]?</p>
<p style="text-align: left;">This has been a very enjoyable conference <img src='http://eddiema.ca/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/iASqdxMcILs" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/12/03/my-talk-at-barcode-of-life-adelaide-2011/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/12/03/my-talk-at-barcode-of-life-adelaide-2011/</feedburner:origLink></item>
		<item>
		<title>Display page headings without page numbers (LaTeX)</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/vj_yZlhAHEM/</link>
		<comments>http://eddiema.ca/2011/11/11/display-page-headings-without-page-numbers-latex/#comments</comments>
		<pubDate>Fri, 11 Nov 2011 14:03:41 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Pure Markup]]></category>
		<category><![CDATA[Elsevier]]></category>
		<category><![CDATA[Headers]]></category>
		<category><![CDATA[LaTeX]]></category>
		<category><![CDATA[linkedin]]></category>
		<category><![CDATA[Omit Page Numbers]]></category>
		<category><![CDATA[Page Numbers]]></category>
		<category><![CDATA[Template]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=3938</guid>
		<description><![CDATA[Brief: While I was preparing for the Complex Adaptive Systems 2011 in Chicago, I bumped into a little problem. The Elsevier LaTeX template for journal papers renders page numbers, but page numbers were not to be included in the final revision for the conference proceedings. We needed to (1) remove page numbers, but (2) continue [...]]]></description>
			<content:encoded><![CDATA[<p><em><strong>Brief:</strong></em> While I was preparing for the Complex Adaptive Systems 2011 in Chicago, I bumped into a little problem. The Elsevier LaTeX template for journal papers renders page numbers, but page numbers were not to be included in the final revision for the conference proceedings.</p>
<p>We needed to (1) remove page numbers, but (2) continue to include page headers that have the paper title and author names.</p>
<p>To do this, I mashed together the horrible block of markup below <img src='http://eddiema.ca/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<pre class="brush: latex; title: ; notranslate">
% declarations for front matter

%%%%% hack -- remove page numbers
\pagestyle{empty}
\usepackage{fancyhdr}
\lhead{}
\chead{\emph{\footnotesize
Primary Author and Secondary Author / Procedia Computer Science 00 (2011)
}}
\cfoot{}
\renewcommand{\headrulewidth}{0pt}
\pagestyle{fancy}
%%%%%

\begin{document}
</pre>
<p>The above example requires the fancy header package (\usepackage{fancyhdr}), we get an italicized, small text header (\emph{\footnotesize{&#8230;}}) with arbitrary text. Since I don&#8217;t explicitly define page numbers, they won&#8217;t appear. We also don&#8217;t need footers (\cfoot{}).</p>
<p>Note that you can create your own arbitrary headers with this package, or &#8212; if you&#8217;re in a rush like I was &#8212; just use the above as is, and replace the arbitrary text <img src='http://eddiema.ca/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<p>Hope this helps!</p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/vj_yZlhAHEM" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/11/11/display-page-headings-without-page-numbers-latex/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/11/11/display-page-headings-without-page-numbers-latex/</feedburner:origLink></item>
		<item>
		<title>My talk at Complex Adaptive Systems, Chicago (2011)</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/6_mqwhKUhic/</link>
		<comments>http://eddiema.ca/2011/11/04/my-slideshow-at-complex-adaptive-systems-chicago/#comments</comments>
		<pubDate>Fri, 04 Nov 2011 04:13:47 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Featured]]></category>
		<category><![CDATA[Machine Learning]]></category>
		<category><![CDATA[CAS]]></category>
		<category><![CDATA[Charlie Obimbo]]></category>
		<category><![CDATA[Complex Adaptive Systems]]></category>
		<category><![CDATA[Evolutionary Computation]]></category>
		<category><![CDATA[Genetic Algorithm]]></category>
		<category><![CDATA[Haochen Zhen]]></category>
		<category><![CDATA[Harmony Search]]></category>
		<category><![CDATA[linkedin]]></category>
		<category><![CDATA[Tiny Encryption Algorithm]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=4158</guid>
		<description><![CDATA[I&#8217;ve just returned from the Complex Adaptive Systems conference on Wednesday after an eight hour drive &#8212; well, most of the driving was done by Dr. Obimbo and Haochen. I presented my paper An evolutionary computation attack on one-round TEA. This paper is built on top of my course project in Computer Security (University of [...]]]></description>
			<content:encoded><![CDATA[<p>I&#8217;ve just returned from the Complex Adaptive Systems conference on Wednesday after an eight hour drive &#8212; well, most of the driving was done by Dr. Obimbo and Haochen. I presented my paper <em>An evolutionary computation attack on one-round TEA</em>. This paper is built on top of my course project in Computer Security (University of Guelph, Winter 2011). This is my first cryptanalysis paper, and is an aside to the bioinformatics focus of my thesis.</p>
<p>My slides introduce Tiny Encryption Algorithm (TEA) pretty well, along with Genetic Algorithm (GA) and Harmony Search (HS). The slides detailing the results aren&#8217;t quite as self-explanatory, but are bearable since the theme is fairly easy to establish: <em>simpler keys are easier to break than more complicated ones</em>.</p>
<p><span style="font-family: helvetica;">&gt;&gt;&gt; <strong>Download: </strong><a href="http://eddiema.ca/wp-content/uploads/2011/11/CAS2011Chicago.pdf">CAS2011Chicago.pdf</a> &#8211; my presentation from Chicago. &lt;&lt;&lt;</span></p>
<div style="text-align:center"><img class="size-full wp-image-4164" title="CAS2011Chicago.005" src="http://eddiema.ca/wp-content/uploads/2011/11/CAS2011Chicago.005.jpg" alt="" width="230" height="172" /><img class="size-full wp-image-4163" title="CAS2011Chicago.009" src="http://eddiema.ca/wp-content/uploads/2011/11/CAS2011Chicago.009.jpg" alt="" width="230" height="172" /><img class="size-full wp-image-4162" title="CAS2011Chicago.021" src="http://eddiema.ca/wp-content/uploads/2011/11/CAS2011Chicago.021.jpg" alt="" width="230" height="172" /></div>
<p>I <em>might</em> look at cryptanalysis again in future &#8212; but I&#8217;ll certainly use Evolutionary Computation (EC) again. It&#8217;s just too readily available in my toolkit, and is far too easy to deploy. One of the major lessons of this project that became very clear through during discussion with the audience is that the operators that are part of an EC algorithm should reflect the kind of problem we&#8217;re trying to solve. This might seem obvious at first, but I think it&#8217;s more subtle than that. For this project, HS enabled the EC to probe a keyspace with many repetitions &#8212; something that GA operators alone didn&#8217;t provide us.</p>
<p>In general however, the solution space is lumpy enough that using ECs against stronger encryption schemes is just not viable &#8212; unless the EC had some magic to overcome the linearly inseparable lumps. I haven&#8217;t yet met such an operator and am not convinced one way or another about its existence. I&#8217;ll certainly introduce you if I ever do bump into it <img src='http://eddiema.ca/wp-includes/images/smilies/icon_razz.gif' alt=':P' class='wp-smiley' /> </p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/6_mqwhKUhic" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/11/04/my-slideshow-at-complex-adaptive-systems-chicago/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/11/04/my-slideshow-at-complex-adaptive-systems-chicago/</feedburner:origLink></item>
		<item>
		<title>Partial Derivatives for Residuals of the Gaussian Function</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/NNLasrJUMlg/</link>
		<comments>http://eddiema.ca/2011/10/10/partial-derivatives-for-residuals-of-the-gaussian-function/#comments</comments>
		<pubDate>Mon, 10 Oct 2011 15:40:16 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Brain]]></category>
		<category><![CDATA[Featured]]></category>
		<category><![CDATA[Curve Fitting]]></category>
		<category><![CDATA[Gaussian Function]]></category>
		<category><![CDATA[linkedin]]></category>
		<category><![CDATA[Partial Derivatives]]></category>
		<category><![CDATA[Residuals]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=4058</guid>
		<description><![CDATA[I needed to get the partial derivatives for the residuals of the Gaussian Function this week. This is needed for a curve fit I&#8217;ll use later. I completely forgot about Maxima, which can do this automatically &#8212; so I did it by hand (Maxima is like Maple, but it&#8217;s free). I&#8217;ve included my work in this [...]]]></description>
			<content:encoded><![CDATA[
<p>I needed to get the partial derivatives for the residuals of the <a href="http://en.wikipedia.org/wiki/Gaussian_function">Gaussian Function</a> this week. This is needed for a curve fit I&#8217;ll use later. I completely forgot about <a href="http://maxima.sourceforge.net/">Maxima</a>, which can do this automatically &#8212; so I did it by hand (Maxima is like Maple, but it&#8217;s free). I&#8217;ve included my work in this post for future reference. If you want a quick refresh on calculus or a step-by-step for this particular function, enjoy <img src='http://eddiema.ca/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> . The math below is rendered with <a href="http://www.mathjax.org/">MathJax</a>.</p>
<p>The Gaussian Function is given by &#8230;</p>
<p>$$ f(x) = ae^{-\frac{(x-b)^2}{2c^2}} $$</p>
<ul>
<li><em>a</em>, <em>b</em>, <em>c</em> are the curve parameters with respect to which we differentiate the residual function</li>
<li><em>e</em> is Euler&#8217;s number</li>
</ul>
<p>Given a set of coordinates I&#8217;d like to fit (x<sub>i</sub>, y<sub>i</sub>), i ∈ [1, m], the residuals are given by &#8230;</p>
<p>$$ r_i = y_i &#8211; ae^{-\frac{(x_i-b)^2}{2c^2}} $$</p>
<p>We want to get &#8230;</p>
<p>$$ \frac{\partial{r}}{\partial{a}}, \frac{\partial{r}}{\partial{b}}, \frac{\partial{r}}{\partial{c}} $$</p>
<p><span id="more-4058"></span></p>
<p>In each of these partials, we can omit the first term y<sub>i</sub> because it drops to zero immediately when held constant.</p>
<p>Let&#8217;s start with <em>r&#8217;(a)</em> &#8230; (<em>b</em>, <em>c</em>, <em>x, y</em> ∈ <em>k</em>)</p>
<p>\begin{align}<br />
r &amp;= -ae^{-\frac{(x-b)^2}{2c^2}} \\<br />
r &amp;= -ae^{u}, (u \in k) \\<br />
r&#8217; &amp;= -e^{u}u&#8217; \\<br />
\therefore \frac{\partial{r}}{\partial{a}} &amp;= -e^{-\frac{(x-b)^2}{2c^2}}<br />
\end{align}</p>
<p>Now let&#8217;s get <em>r&#8217;(b)</em> &#8230; (<em>a</em>, <em>c</em>, <em>x, y</em> ∈ <em>k</em>)</p>
<p>\begin{align}<br />
r &amp;= -ae^{-\frac{(x-b)^2}{2c^2}} \\<br />
r &amp;= -ae^{u} \\<br />
u &amp;= -\frac{(x-b)^2}{2c^2} \\<br />
u &amp;= -\frac{x^2-2bx+b^2}{2c^2} \\<br />
u &amp;= -\frac{x^2}{2c^2}-\frac{2bx+b^2}{2c^2}, (-\frac{x^2}{2c^2} \in k) \\<br />
u&#8217; &amp;= -\frac{-2x + 2b}{2c^2} \\<br />
u&#8217; &amp;= -\frac{b &#8211; x}{c^2} \\<br />
r&#8217; &amp;= -ae^{u}(u&#8217;) \\<br />
r&#8217; &amp;= -ae^{-\frac{(x-b)^2}{2c^2}}(-\frac{b &#8211; x}{c^2}) \\<br />
\therefore \frac{\partial{r}}{\partial{b}} &amp;= \frac{a(b &#8211; x)e^{-\frac{(x-b)^2}{2c^2}}}{c^2}<br />
\end{align}</p>
<p>Finally, let&#8217;s get <em>r&#8217;(c)</em> &#8230; (<em>a</em>, <em>b</em>, <em>x, y</em> ∈ <em>k</em>)</p>
<p>\begin{align}<br />
r &amp;= -ae^{-\frac{(x-b)^2}{2c^2}} \\<br />
r &amp;= -ae^{u} \\<br />
u &amp;= -\frac{(x-b)^2}{2c^2} \\<br />
u &amp;= -{(x-b)^2}\frac{1}{2c^2}, (-{(x-b)^2} \in k) \\<br />
(\frac{1}{v})&#8217; &amp;= \frac{-v&#8217;}{v^2} \\<br />
(\frac{1}{2c^2})&#8217; &amp;= \frac{-(2c^2)&#8217;}{(2c^2)^2} \\<br />
&amp;= \frac{-4c}{4c^4} \\<br />
&amp;= \frac{-1}{c^3}, (c \neq 0) \\<br />
u&#8217; &amp;= -(x-b)^2\frac{-1}{c^3} \\<br />
u&#8217; &amp;= \frac{(x-b)^2}{c^3} \\<br />
r&#8217; &amp;= -ae^{u}(u&#8217;) \\<br />
r&#8217; &amp;= -ae^{-\frac{(x-b)^2}{2c^2}}(\frac{(x-b)^2}{c^3}) \\<br />
\therefore \frac{\partial{r}}{\partial{c}} &amp;= \frac{-a(x-b)^2e^{-\frac{(x-b)^2}{2c^2}}}{c^3}<br />
\end{align}</p>
<p>That&#8217;s all three partials!</p>
<p>Here are Maxima&#8217;s answers for <em>r&#8217;(b)</em> and <em>r&#8217;(c)</em> &#8230;<br />
<code><br />
dr/db = -a*(x-b)*%e^-((x-b)^2/(2*c^2))/c^2<br />
dr/dc = -a*(x-b)^2*%e^-((x-b)^2/(2*c^2))/c^3<br />
</code></p>
<p> <img src='http://eddiema.ca/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/NNLasrJUMlg" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/10/10/partial-derivatives-for-residuals-of-the-gaussian-function/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/10/10/partial-derivatives-for-residuals-of-the-gaussian-function/</feedburner:origLink></item>
		<item>
		<title>Summer 2011 in Review</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/yR3Dso8rlwA/</link>
		<comments>http://eddiema.ca/2011/08/28/summer-2011-in-review/#comments</comments>
		<pubDate>Sun, 28 Aug 2011 13:06:52 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Life]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=3982</guid>
		<description><![CDATA[Below, the highlights of my summer semester &#8211; Academic &#8211; Thesis The topic area of my thesis is starting to become more concrete. My advisor, Dr. SC Kremer has partnered with Mr. S Ratnasingham of Barcode of Life (BOL), Biodiversity Institute of Canada, in order to launch me into an exciting project. My project is [...]]]></description>
			<content:encoded><![CDATA[<p>Below, the highlights of my summer semester &#8211;</p>
<p><strong>Academic &#8211; Thesis</strong></p>
<p>The topic area of my thesis is starting to become more concrete. My advisor, Dr. SC Kremer has partnered with Mr. S Ratnasingham of Barcode of Life (BOL), Biodiversity Institute of Canada, in order to launch me into an exciting project. My project is a machine learning item that aims to automatically clean up errors in chromatographs (and where possible and practical &#8212; errors introduced by human intelligence). The current step is to set up all of the data <em>accessibly</em> so that both human intelligence and the proposed software will have a shot of making sense of it all. We&#8217;re dealing with a lot (most?) of the data in the entire BOL database, so we expect that our <em>accessible version</em> will take up twelve to twenty gigabytes after derived data and annotation. The motivation is spurred on by a conference deadline that <em>is only just</em> outside of the near future.</p>
<p><span id="more-3982"></span></p>
<p><strong>Extracurricular &#8211; Academic</strong></p>
<p>Working with Dr. R Heck over at Environmental Sciences has given me the chance to <em>load my brain more evenly</em>. At this position, I&#8217;ve been writing plugin software compatible with ImageJ, an open source Java image processing and analysis platform; and to a lesser extent, select features of the Benoit toolbox in MATLAB. This work revolves around various tasks peering into cubes of soil that have been imaged with CT scans (all images are 3D cubes of pixels). I have thus far completed one major piece of software that analyzes the semivariogram statistics of such a cube (the semivariogram or semivariance is a second-moment statistic), and have two additional pieces of software on the way (one that generates <em>idealized</em> images, and another that describes the <em>fractal dimensions</em> of an image). I&#8217;ve also had the chance to work with Apache/MySQL administrative items and have gotten my hands quite dirty with phpBB.</p>
<p>In each of the three courses I took in Winter Semester, I produced final projects that I was quite happy with. One of them came out of Computer Security and Dr. Obimbo was just as happy. My project attacks one-round Tiny Encryption Algorithm (TEA) with an evolutionary computation technique which melts Genetic Algorithm together with Harmony Search. He submitted my project to the Complex Adaptive Systems conference in Chicago where it was reviewed and accepted.</p>
<p><strong>Extracurricular &#8211; Leadership</strong></p>
<p>With Richard Schwarting stepping down as one of the two graduate student representatives at the School of Computer Science council meetings, I&#8217;ve stepped up to take his place and will serve with Jason Ernst. The first order of business is to arrange a nice social event for early Fall to foster a sense of community that many graduate students find a bit lacking. Where there is will, there certainly is a way. The challenge we face is the diversity of the group. What is arguably the greatest asset we have in innovation is also a hurdle when designing something that everyone will enjoy and with which, find inclusion. So far, the most enjoyable activities I&#8217;ve heard suggested are (1) bowling, (2) a free form sporadic arrangement of board games all running concurrently, and (3) food tasting with care toward individuals&#8217; sensitivities.</p>
<p>Jason and the infamous Joseph Brown have also roped me in to get the ACM Guelph chapter website operational. Because of the workload, I think that the best way about this is to recruit some undergraduate help in producing WordPress and MediaWiki compatible content while I administer things like installations and permissions. This delegation of content generation offers undergraduates an opportunity to increase their exposure to the societies such as the ACM and is likely useful as a resume piece.</p>
<p><strong>Physical Fitness</strong></p>
<p>From the end of July up until today, I have made it a point of myself to become more physically fit. I&#8217;ve managed to repair the amount of swimming I can do up from thirty minutes to 96 lengths of a 25m pool in an hour. My goal is 100 lengths before Fall semester with the addition that I meet the sixty-second 100m head-up sprint requirement in the new lifeguarding standards to be rolled out over the next two years. The other new standards will have to wait as I&#8217;ve learned that pacing is quite important. Where I&#8217;ve had time, I&#8217;ve inserted 30 or 60 minute sessions on the performance-resistance-12 setting of the ellipticals. For lifting, I&#8217;m still stuck in the neuromuscular phase where everything is getting rewired. My coordination is coming back, and I have noted some physical effects in general health (faster to wake, easier to cold-start sprint, tolerance to temperature changes, better breath control etc.). According to the text books, I have two more weeks in the neuromuscular phase before I can start putting on the heavier weights. The most useful advice I&#8217;ve learned from this is (1) to avoid exercise machines for the upper body, as one does not develop the stabilizing muscle groups across joints and (2) to lift weights slowly in order to properly use weight as resistance, rather than momentum as assistance.</p>
<p>Finally, I&#8217;ve put my renewed fitness to good work with a recertification in the National Lifeguarding Service (NLS) as a lifeguard. I&#8217;ve chosen to get the latest version of Standard First Aid + CPR-C as well in July and will be packing in Automated External Defibrillation on Monday. I might get Airway Management some time in Fall if time and budget permit &#8212; it&#8217;s not a requirement for lifeguarding, but it <em>is</em> good to know since I think oxygen tanks and bag-valve-masks are pretty neat devices.</p>
<p><strong>Rolling into Fall</strong></p>
<p>I think that about sums up my summer. Clearly, items will shift in importance and time dedication when the Fall semester rolls in along with the de facto academic year. I intend however to keep my brain working well by evenly loading it, and to keep my body working properly in order to support my brain. Cheers!</p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/yR3Dso8rlwA" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/08/28/summer-2011-in-review/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/08/28/summer-2011-in-review/</feedburner:origLink></item>
		<item>
		<title>Reverse Complement (C Snippet)</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/U6TLYvxcnmM/</link>
		<comments>http://eddiema.ca/2011/07/17/reverse-complement-c-snippet/#comments</comments>
		<pubDate>Mon, 18 Jul 2011 01:20:34 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Computational Biology]]></category>
		<category><![CDATA[Reverse Complement]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=3916</guid>
		<description><![CDATA[Brief: If you&#8217;ve arrived by Google for code to reverse complement a DNA sequence in C, here you go. I&#8217;ve written this so many times, I&#8217;ve decided to save a copy here. The above implementation is released to the public domain &#8212; i.e. do whatever you want with it. Usage: The compiled code reverse complements the [...]]]></description>
			<content:encoded><![CDATA[<p><strong><em>Brief:</em></strong> If you&#8217;ve arrived by Google for code to reverse complement a DNA sequence in C, here you go. I&#8217;ve written this so many times, I&#8217;ve decided to save a copy here.</p>
<pre class="brush: cpp; title: ; notranslate">
#include &lt;stdio.h&gt;
#include &lt;string.h&gt;
int main(int argc, char** argv) {
    if(argc &lt; 2) {
        fprintf(stderr, &quot;&gt;&gt;&gt; Need an argument to reverse complement.\n&quot;);
        return 0;
    }
    int last = strlen(argv[1]) - 1;
    for(int i = last; i &gt;= 0; i --) {
        switch(argv[1][i]) {
            case 'G':
                putc('C', stdout);
                break;
            case 'A':
                putc('T', stdout);
                break;
            case 'T':
                putc('A', stdout);
                break;
            case 'C':
                putc('G', stdout);
                break;
            default:
                putc(argv[1][i], stdout);
                break;
        }
    }
    putc('\n', stdout);
    return 0;
}
</pre>
<p><em>The above implementation is released to the public domain &#8212; i.e. do whatever you want with it.</em></p>
<p><em><strong>Usage:</strong></em> The compiled code reverse complements the uppercase characters of an argument given to it. If you give it lowercase characters or characters other than {A, C, G, T}, it just pumps it back out.</p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/U6TLYvxcnmM" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/07/17/reverse-complement-c-snippet/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/07/17/reverse-complement-c-snippet/</feedburner:origLink></item>
		<item>
		<title>Exposing MATLAB data with JFrame &amp; JTextArea</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/Ri0GXd3gmts/</link>
		<comments>http://eddiema.ca/2011/07/11/exposing-matlab-data-with-jframe-jtextarea/#comments</comments>
		<pubDate>Mon, 11 Jul 2011 05:18:55 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Featured]]></category>
		<category><![CDATA[Pure Programming]]></category>
		<category><![CDATA[Java]]></category>
		<category><![CDATA[JFrame]]></category>
		<category><![CDATA[JTextArea]]></category>
		<category><![CDATA[linkedin]]></category>
		<category><![CDATA[MATLAB]]></category>
		<category><![CDATA[swing]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=3882</guid>
		<description><![CDATA[Today, I&#8217;ll describe my first foray into MATLAB. My task is simple &#8212; there&#8217;s already a GUI developed as a toolbox. The user inputs a bunch of data, and it crunches it away and provides a graph and displays a dialogue box about the moments of the data. The issue here is that we&#8217;d like [...]]]></description>
			<content:encoded><![CDATA[<p>Today, I&#8217;ll describe my first foray into MATLAB. My task is simple &#8212; there&#8217;s already a GUI developed as a toolbox. The user inputs a bunch of data, and it crunches it away and provides a graph and displays a dialogue box about the moments of the data. The issue here is that we&#8217;d like to know more about the data &#8212; particularly, we&#8217;d like to retrieve the ordered pairs corresponding to each of the plotted points in the graph.</p>
<p>In this exercise, I&#8217;ll show you how to use JFrame &amp; JTextArea to display the coordinates of a graph in an already existing MATLAB GUI toolbox or plugin.</p>
<p><em>The reason why I&#8217;ve decided on this approach rather than outputting directly to the MATLAB console is because I eventually want to add additional functions to reshape and reformat the text, and also to save the text that appears in its own window using additional Java swing components. But that&#8217;s a story for another day.</em></p>
<p><strong>The Quick How-To &#8230; ( </strong><em>3 steps</em><strong> )</strong></p>
<p>Choose the toolbox or plugin you&#8217;d like to modify and open up its &#8220;.m&#8221; file. This is where you&#8217;ll want to add your custom code. There are three parts to this exercise &#8212; first, we need to get a few classes from Java; then, we need to store the data we want to display; finally, we make the display appear.</p>
<p>In this example, I&#8217;ll import the bare minimal amount of Java classes &#8212; you can extend this functionality by adding more swing classes if you like. I&#8217;ve arbitrarily prefixed my Java variables above with the prefix &#8220;expose&#8221;.</p>
<pre class="brush: java; title: ; notranslate">
import javax.swing.JFrame;
import javax.swing.JTextArea;
import javax.swing.JScrollPane; // Scrollbars -- optional.
exposeReport = JFrame('Data Report');
exposeTa = JTextArea(24, 80);
exposeScroll = JScrollPane(exposeTa);
exposeScroll.setVerticalScrollBarPolicy(JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
exposePane = exposeReport.getContentPane();
exposePane.add(exposeScroll);
exposeReport.pack();
</pre>
<p><em>I&#8217;m not exactly sure why MATLAB doesn&#8217;t include scrollbars automatically.</em></p>
<p>Of the above, there are only two variables which we&#8217;ll need to refer to again later &#8212; &#8220;exposeReport&#8221; and &#8220;exposePane&#8221;.</p>
<p>The next step is to use your JTextArea as an output terminal. Just append strings to it as the graph is being built &#8212; you&#8217;ll have to look at your specific plugin to figure out the logic behind the graph &#8212; in general, you&#8217;ll be looking for a for-loop and a function that alters the graph.</p>
<pre class="brush: java; title: ; notranslate">
// Look for a 'for' loop with a line that adds data to the graph.
// Here, I've used the variable SomeX for a point's x-coordinate,
// and SomeY for a point's y-coordinate.
exposeTa.append(sprintf('%d\t%d\n', SomeX, SomeY));
</pre>
<p>The loop spins around and incrementally adds all of the points. Note that I&#8217;ve used &#8220;%d&#8221; as a conversion to numbers expressed in base-ten (including floating point values). This is different from the conversion character in C where &#8220;%d&#8221; indicates only integers.</p>
<p>We add the final code after the loop exits. This next code makes your data report window visible &#8212; just as you&#8217;d expect from Java.</p>
<pre class="brush: java; title: ; notranslate">
exposeReport.setVisible(1);
</pre>
<p>I opted to put this line after the code that reveals the toolbox&#8217;s original summary window &#8212; this causes my report to appear on top.</p>
<p>That&#8217;s all there is to it! Enjoy <img src='http://eddiema.ca/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<p><strong>MATLAB from a CS student perspective</strong></p>
<p><strong></strong>MATLAB has always been a bit of an oddball language for me. It&#8217;s dynamically typed but also exposes Java&#8217;s classes if you ask nicely. It&#8217;s C-looking and provides functions that the C-programmer would be happy accepting such as <em>strcat</em> and <em>sprintf</em>, but again &#8212; putting on the dynamic spin by returning the constructed string rather than modifying the contents of a buffer. All in all, the design choices do a good job of making MATLAB do what it&#8217;s intended to do; it gives scientists and engineers a way to succinctly express math to a machine without having to take too many computer science courses.</p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/Ri0GXd3gmts" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/07/11/exposing-matlab-data-with-jframe-jtextarea/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/07/11/exposing-matlab-data-with-jframe-jtextarea/</feedburner:origLink></item>
		<item>
		<title>Blog Author Clustering with SOMs</title>
		<link>http://feedproxy.google.com/~r/eddiema/~3/twMG9pDq2c0/</link>
		<comments>http://eddiema.ca/2011/06/29/blog-author-clustering-with-soms/#comments</comments>
		<pubDate>Wed, 29 Jun 2011 18:10:40 +0000</pubDate>
		<dc:creator>Eddie Ma</dc:creator>
				<category><![CDATA[Featured]]></category>
		<category><![CDATA[Machine Learning]]></category>
		<category><![CDATA[Blog]]></category>
		<category><![CDATA[Kohonen]]></category>
		<category><![CDATA[Lexicon]]></category>
		<category><![CDATA[linkedin]]></category>
		<category><![CDATA[Neural Networks]]></category>
		<category><![CDATA[Self-Organizing Map]]></category>
		<category><![CDATA[SOM]]></category>

		<guid isPermaLink="false">http://eddiema.ca/?p=3478</guid>
		<description><![CDATA[For the final project in Neural Networks (CIS 6050 W11), I decided to cluster blog posts based on the difference between an author&#8217;s word choice and the word choice of the entire group of authors. &#62;&#62;&#62; Attached Course Paper: Kohonen Self-Organizing Maps in Clustering of Blog Authors (pdf) &#60;&#60;&#60; A self-organizing map (Kohonen SOM) strategy was used. The [...]]]></description>
			<content:encoded><![CDATA[<p>For the final project in Neural Networks (CIS 6050 W11), I decided to cluster blog posts based on the difference between an author&#8217;s word choice and the word choice of the entire group of authors.</p>
<p>&gt;&gt;&gt; <strong>Attached Course Paper:</strong> <a href="http://eddiema.ca/wp-content/uploads/2011/06/Kohonen-Self-Organizing-Maps-in-Clustering-of-Blog-Authors.pdf">Kohonen Self-Organizing Maps in Clustering of Blog Authors</a> (pdf) &lt;&lt;&lt;</p>
<p>A self-organizing map (Kohonen SOM) strategy was used. The words chosen to compose a given blog post defined wherein the map it should be placed. The purpose of this project was to figure out what predictive power could be harnessed given the combination of the SOM and each author&#8217;s lexicon; i.e. whether or not it is possible to automatically categorize an author&#8217;s latest post without the use of any tools besides the above.</p>
<p><strong><em>Data:</em></strong> Thirteen authors contributing a total of fourteen blogs participated in the study (all data was retrieved on 2011 March 8th). The below table summarizes the origin of the data.</p>
<table style="margin-left: auto; margin-right: auto;" border="0" cellspacing="0" cellpadding="0">
<tbody>
<tr style="background-color: #000000;">
<td style="padding-left: 5px; padding-right: 5px;"><span style="color: #ffffff;"><strong>Author</strong></span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="color: #ffffff;"><strong>Posts</strong></span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="color: #ffffff;"><strong>Lexicon</strong></span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="color: #ffffff;"><strong>Blog Name</strong></span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="color: #ffffff;"><strong>Subject Matter</strong></span></td>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">Andre Masella</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">198</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">7953</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://www.masella.name/">MasellaSphere</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Synth Bio, Cooking, Engineering</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 5px; padding-right: 5px;">Andrew Berry</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">46</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">2630</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://www.abdevelopment.ca/">Andrew Berry Development</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Drupal, Web Dev, Gadgets</td>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">Arianne Villa</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">41</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">1217</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://mrfmr.tumblr.com/">&#8230;mrfmr.</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Internet Culture, Life</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 5px; padding-right: 5px;">Cara Ma</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">12</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">854</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://cara.blogdns.com/">Cara&#8217;s Awesome Blog</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Life, Pets, Health</td>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">Daniela Mihalciuc</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">211</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">4454</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://www.citisenoftheworld.blogspot.com/">Citisen of the World</a><sup>†</sup></td>
<td style="padding-left: 5px; padding-right: 5px;">Travel, Life, Photographs</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 5px; padding-right: 5px;">Eddie Ma</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">161</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">5960</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://eddiema.ca">Ed&#8217;s Big Plans</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Computing, Academic, Science</td>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">Jason Ernst</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">61</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">3445</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://www.jasonernst.com/">Jason&#8217;s Computer Science Blog</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Computing, Academic</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 5px; padding-right: 5px;">John Heil</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">4</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">712</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://jheil.wordpress.com/">Dos Margaritas Por Favor</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Science, Music, Photography</td>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">Lauren Stein</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">91</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">4784</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://laurenbstein.wordpress.com/blog/">The Most Interesting Person</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Improv, Happiness, Events</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 5px; padding-right: 5px;">Lauren Stein (Cooking)</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">7</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">593</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://thelaurentinacookbook.wordpress.com/">The Laurentina Cookbook</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Cooking, Humour</td>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">Liv Monck-Whipp</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">30</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">398</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://omniology.blogspot.com/">omniology</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Academic, Biology, Science</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 5px; padding-right: 5px;">Matthew Gingerich</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">98</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">395</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://blog.majugi.com/">The Majugi Blog</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Academic, Synth Bio, Engineering</td>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">Richard Schwarting</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">238</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">7538</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://blog.kosmokaryote.org/">Kosmokaryote</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Academic, Computing, Linux</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 5px; padding-right: 5px;">Tony Thompson</td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">51</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><span style="font-family: 'andale mono', times;">2346</span></td>
<td style="padding-left: 5px; padding-right: 5px;"><a href="http://www.tony.thompson.name/">Tony Thompson, Geek for Hire</a></td>
<td style="padding-left: 5px; padding-right: 5px;">Circuitry, Electronic Craft, Academic</td>
</tr>
</tbody>
</table>
<p>†<em>Daniela remarks that the spelling of <span style="text-decoration: underline;">Citisen</span> is intentional.</em></p>
<p>In order to place the blog posts into a SOM, each post was converted to a bitvector. Each bit is assigned to a specific word, so that the positions of each bit consistently represents the same word from post to post. An on-bit represented the presence of a word while an off-bit represented the absence of a word. Frequently used words like &#8220;the&#8221; were omitted from the word bit-vector, and seldom used words were also omitted.</p>
<p><img style="margin: 0px; border: 0px; float: left;" title="Self Organizing Map (SOM) -- All Maps" src="http://eddiema.ca/wp-content/uploads/2011/04/som_all2.png" alt="" width="312" height="599" /><strong><em>Results:</em></strong> The center image (in the collection to the left) is a density map where darker pixels indicates a larger number of posts &#8212; this centre map represents all of the posts made by all of the authors pooled together.</p>
<p>Because of the number of posts and the number of authors, I&#8217;ve exploded the single SOM image into the remaining fourteen images.</p>
<p>It was found that posts were most often clustered together if they were both by the same author and on the same topic. Clusters containing more than one author generally did not show much agreement about the topic.</p>
<p>Regions of this SOM were dominated by particular authors and topics as below.</p>
<table style="margin-left: auto; margin-right: auto; padding-bottom: 24px;" border="0" cellspacing="0" cellpadding="0">
<tbody>
<tr style="background-color: #000000;">
<td style="padding-left: 8px; padding-right: 8px;"><strong><span style="color: #ffffff;">Region</span></strong></td>
<td style="padding-left: 8px; padding-right: 8px;"><strong><span style="color: #ffffff;">Authors</span></strong></td>
<td style="padding-left: 8px; padding-right: 8px;"><strong><span style="color: #ffffff;">Topics</span></strong></td>
</tr>
<tr>
<td style="padding-left: 8px; padding-right: 8px;" rowspan="3">Top Left</td>
<td style="padding-left: 8px; padding-right: 8px;">Liv</td>
<td style="padding-left: 8px; padding-right: 8px;">Academic Journals</td>
</tr>
<tr>
<td style="padding-left: 8px; padding-right: 8px;">Eddie</td>
<td style="padding-left: 8px; padding-right: 8px;">Software Projects</td>
</tr>
<tr>
<td style="padding-left: 8px; padding-right: 8px;">Jason</td>
<td style="padding-left: 8px; padding-right: 8px;">Academic</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;" rowspan="2">Top Border</td>
<td style="padding-left: 8px; padding-right: 8px;">Lauren</td>
<td style="padding-left: 8px; padding-right: 8px;">Human Idiosyncrasies</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;">Richard</td>
<td style="padding-left: 8px; padding-right: 8px;">Linux</td>
</tr>
<tr>
<td style="padding-left: 8px; padding-right: 8px;">Top Right</td>
<td style="padding-left: 8px; padding-right: 8px;">Lauren</td>
<td style="padding-left: 8px; padding-right: 8px;">Improv</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;">Up &amp; Left of Centre</td>
<td style="padding-left: 8px; padding-right: 8px;">Daniela</td>
<td style="padding-left: 8px; padding-right: 8px;">Travel</td>
</tr>
<tr>
<td style="padding-left: 8px; padding-right: 8px;">Centre</td>
<td style="padding-left: 8px; padding-right: 8px;"><em>all</em></td>
<td style="padding-left: 8px; padding-right: 8px;"><em>short and misfit posts</em></td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;">Right Border</td>
<td style="padding-left: 8px; padding-right: 8px;">Andre</td>
<td style="padding-left: 8px; padding-right: 8px;">Cooking</td>
</tr>
<tr>
<td style="padding-left: 8px; padding-right: 8px;">Just Below Centre</td>
<td style="padding-left: 8px; padding-right: 8px;">Matthew</td>
<td style="padding-left: 8px; padding-right: 8px;">Software Projects</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;" rowspan="3">Bottom Left</td>
<td style="padding-left: 8px; padding-right: 8px;">Andre</td>
<td style="padding-left: 8px; padding-right: 8px;">Language Theory</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;">Andrew</td>
<td style="padding-left: 8px; padding-right: 8px;">Software Projects</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;">Jason</td>
<td style="padding-left: 8px; padding-right: 8px;">Software Projects</td>
</tr>
<tr>
<td style="padding-left: 8px; padding-right: 8px;">Bottom Border</td>
<td style="padding-left: 8px; padding-right: 8px;">Richard</td>
<td style="padding-left: 8px; padding-right: 8px;">Academic</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;" rowspan="2">Bottom Right</td>
<td style="padding-left: 8px; padding-right: 8px;">Eddie</td>
<td style="padding-left: 8px; padding-right: 8px;">Web Development</td>
</tr>
<tr style="background-color: #e0e8ee;">
<td style="padding-left: 8px; padding-right: 8px;">Jason</td>
<td style="padding-left: 8px; padding-right: 8px;">Software Tutorials</td>
</tr>
</tbody>
</table>
<p><strong><em>Discussion:</em></strong> There are some numerical results to go along with this, but they aren&#8217;t terribly exciting &#8212; the long and the short of it is that this project should to be repeated. The present work points towards the following two needed improvements.</p>
<p>First, the way the bitvectors were cropped at the beginning and at the end were based on a usage heuristic that doesn&#8217;t really conform to information theory. I&#8217;d likely take a look at the positional density of all bits to select meaningful words to cluster.</p>
<p>Second, all posts were included &#8212; this results in the dense spot in the middle of the central map. Whether these posts are short or just misfit, many of them can probably be removed by analyzing their bit density too.</p>
<p><strong><em>Appendix:</em></strong> Here are two figures that describe the distribution of the data in the word bitvectors.</p>
<table style="margin-left: auto; margin-right: auto;" border="0" cellspacing="0" cellpadding="0">
<tbody>
<tr>
<td><img class="alignnone size-full wp-image-3797" title="word_count_in_ranks" src="http://eddiema.ca/wp-content/uploads/2011/06/word_count_in_ranks1.png" alt="" width="355" height="163" /></td>
<td><img class="alignnone size-full wp-image-3798" title="word_uses_in_ranks" src="http://eddiema.ca/wp-content/uploads/2011/06/word_uses_in_ranks1.png" alt="" width="354" height="163" /></td>
</tr>
</tbody>
</table>
<p>When we sort the words based from a high number of occurrences down to a low number of occurrences, we get graphs that look like the above two. A rank class contains all words that have the same number of occurrences across the entire study. The impulse graph on the left shows the trend for the <em><span style="text-decoration: underline;">number of unique words</span></em> in each rank class. The number of words drastically increases as the classes contain fewer words. The impulse graph on the right shows the trend for the <em><span style="text-decoration: underline;">count of uses</span></em> for words in a given rank class. The number of uses decreases as words become more rare.</p>
<p>These graphs were made before the main body of the work to sketch out how I wanted the bitvectors to behave &#8212; they verify that there was nothing unusual about the way the words were distributed amongst the data.</p>
<img src="http://feeds.feedburner.com/~r/eddiema/~4/twMG9pDq2c0" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://eddiema.ca/2011/06/29/blog-author-clustering-with-soms/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		<feedburner:origLink>http://eddiema.ca/2011/06/29/blog-author-clustering-with-soms/</feedburner:origLink></item>
	</channel>
</rss>

