About Jan-Marten de Boer

Jan-Marten de Boer is a self employed web entrepeneur. He made his debut on the web with grace and registered Joh Man X at age 18. With an ever growing skill set and curiosity towards web technology, he specializes in server side scripting and UI/UX through HTML5, CSS3 and the glorious JavaScript.

Rating companies using Wufoo forms

One of my clients asked me to make them a simple site with a landing page and a Wufoo form. The landing page contains a brief story about the website, some local news and a local weather report. (hooever.nl)

The Wufoo form is just a simple one, asking questions about the experience customers had with their company. In this small survey, the visitor rates the experience using likert fields.

Now my customer wants to crunch the numbers from the survey and give the company a grade, using the Dutch grading system, seen in High school, rating from 1-10. I thought this was a simple request, but it turns out I really need to digest a number of structural elements using the Wufoo RESTfull API. I just wanted to share the magic with you guys, so we can all benefit from my research ;) In reality I store this result in a cache, based on the result of /api/v3/forms/hash/entries/count.json.

I’m really gratefull for @chriscoyier writing about Wufoo or else I’d have ended up making my own malicious platform for forms.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
 
<?php
 
class wufoo{
 
    private $api_version = 'v3';
    private $user;
    private $pass = 'footastic';
    private $subdomain;
 
    function __construct($user,$subdomain,$api_version='v3')
        {
            $this->user = $user;
            $this->subdomain = $subdomain;
            $this->api_version = $api_version;
        }
 
    public function request($doc,$params=null)
        {
            $params = ($params!==null) ? ltrim($params,'&?') : '';
            $url = 'https://'.$this->subdomain.'.wufoo.com/api/'.trim($this->api_version,'/').'/'.$doc.'.json'.$params;
 
            $ch = curl_init($url);
            curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
            curl_setopt($ch,CURLOPT_USERPWD,$this->user.':'.$this->pass);
            curl_setopt($ch,CURLOPT_HTTPAUTH,CURLAUTH_ANY);
            curl_setopt($ch,CURLOPT_SSL_VERIFYPEER,false);
            curl_setopt($ch,CURLOPT_FOLLOWLOCATION,true);
            curl_setopt($ch,CURLOPT_USERAGENT,'Joh Man X: Wufoo API agent');
 
            $response = curl_exec($ch);
            $status = curl_getinfo($ch);
 
            if($status['http_code']==200){return json_decode($response);}
            return false;
        }
 
    public function form($form)
        {
            return $this->request('forms/'.$form);
        }
 
    public function form_fields($form)
        {
            return $this->request('forms/'.$form.'/fields');
        }
 
    public function form_entries($form)
        {
            return $this->request('forms/'.$form.'/entries');
        }
}
 
$form_hash = 'xxxxxx';
$wufoo = new wufoo('XXXX-XXXX-XXXX-XXXX','subdomain');
 
$fields = $wufoo->form_fields($form_hash);
$entries = $wufoo->form_entries($form_hash);
 
$field_a = array();
foreach($fields->Fields AS $field)
    {
        if($field->Type=='likert')
            {
                $choices = array();
                foreach($field->Choices AS $choice)
                    {
                        $choices[$choice->Label] = (int) $choice->Score;
                    }
                foreach($field->SubFields AS $subfield)
                    {
                        $field_a[$subfield->ID] = $choices;
                    }
            }
    }
 
$score = 0;
foreach($entries->Entries AS $entry)
    {
        $entryscore = 0;
        $usable = 0;
        foreach($entry AS $option=>$value)
            {
                if(isset($field_a[$option])&&!empty($value))
                    {
                        $oscore = $field_a[$option][$value];
                        $tscore = (in_array(0,$field_a[$option])) ? count($field_a[$option])-1 : count($field_a[$option]); // 0==not applicable
                        $oscore /= $tscore;
                        $entryscore += $oscore;
                        $usable++;
                    }
            }
        $entryscore /= $usable;
        $score += $entryscore;
    }
$score /= count($entries->Entries);
$score = round(($score*9)+1,1);
 
echo $score;
 
?>

Dutch coin made with QR code

“De Nederlandsche Munt” is the first in the world to forge a coin with a QR code on it. The code is a “symbool van het hedendaagse leven” symbol to present day life.

The QR code can be found on the back side of the coin. It’s been forged today at 12AM local time (GMT+0200). Scanning the code results in reading a link to a site with information about the coin. (Damn, I wanted it to be a means of NFC payment)

It’s the first time in world history for a coin to be equipped with a QR code. It unraveled quite an interest in our tech society. Scanning the code with my G1 (HTC Dream; they still exist) resulted in the link q5g.nl and luckily made me aware of q5g.nl/en/ .

“Technology and the internet have become a solid part of our lifes. A world in which the master-pupil concept changes in terms of exchanging knowledge. This is mainly the result of the internet being the main source of information and backgrounds. A world in which coins are no longer made with hammers, but with the use of laser technology. A world of QR codes, augmented reality, internet, screens, lasers and mobile phones.” – Freely translated by me, from Dutch to English. Origin unknown.

Surprisingly, when I open the site on my Android device, it’s clear that the site actually has no proper optimization for a mobile browser. This is really odd, since QR codes are mostly used by mobile phones. Maybe something I should look into.

 

2011, the year of smart

Everyone wants a piece of the action nowadays. Even though we actually can’t afford a €700 device for reading our news or calling mum, we all wanna have one. I must admit, although I’m still quite fond of my G1 (released 2008) and my Acer Aspire One netbook (they released as early as 2007), we’re already at 2011 and not much has changed for consumers. Or has it?

They call ‘m pads and tabs

Oh yeah, those devices. While we’ve had a great predecessor to them, the Dell XPS Tablets, they start at a price of $2000 USD. That’s a reasonable price for a full fledged workstation replacement with a flipping touchscreen that allows writing, but not nearly in the price range of a day to day consumer. And as marketing always prescribes: “Gotta catch ‘m all!”

DellXPS-tablet

The generation of “smart” device we’re looking at now is befitting the category that had to be made for the iPad and Galaxy tab from resp. Apple and Samsung. Some call it pads, others tabs and there are even those who tend to fit them into “smart devices”. Personally I think that’s going to far, because that would also fit smartphones, netbooks and possibly even notebooks.

I heard about this “cloud”-thingy back in 2010

While cloud computing as a concept has existed for some time already, it began to take shape late ’09 and during ’10. The concept is simple. Take a whole bunch of computing capacity. Make it open for public and provide the public with services, so their computer can be simple, while the cloud solves difficult problems like computational rounds and scaling storage and bandwith. To represent this simple, you can take the following example:

Johnny needs to work on a school project. He’s working at home, in the train from and to school, at school and even at his friend Jake’s place. Now, normally we would go about installing an office suite on all those computers and storing the documents on a flash drive, so he can work on the same document in the same office suite.

There comes Google Docs. Johnny heard about this back in 2009. It’s supposed to be really easy to use and he doesn’t need to store his documents on a flash drive he can lose. He also doesn’t need to buy an expensive office license for each computer. It all works from within his web browser of choice and nowadays all desktop operating systems are packed with one of those. With just a username and password, Johnny can access all his documents, wherever he goes and he doesn’t need to think about making backups, because Google stores his document in lots of places, all over the globe.

What’s got cloud to do with smart devices?

While at first this may look as two completely different things; cloud computing and smart devices go well hand in hand. While smart devices can do a lot, their capabilities are fairly limited, due to cutting on production costs. If we were to combine cloud and a smart device, we would be able to buy cheap travel mates that work well without a whole lot of power, because we leave that to the cloud. Now who doesn’t say yes to having more for less?

Actually, this company petitpetit (ptpt) came with just the solution. They developed a netbook that costs nearly zip and works as a full fledged personal computer, due to the connection to the cloud. I must point out, I’m fairly sure you can’t throw away your gaming PC or Designing PC after buying one of those, but ah well. There’s a short video about how the concept works and how the netbook looks like. Enjoy!

Creating an HTTP proxy for Wikipedia

How often is it that we want to clarify our text by explaining our terms, so most of our readers actually understand what we’re saying and how often is it that we end up with an article that’s either bloated with explenational lyrics to a song called “The article I was supposedly writing” or an article far too in-depth for the common reader. A lot of people stop reading when they find out they have to read a Wiki article every five sentences, but others really love that and spend their late hours on browsing through all that extended information.

I myself came up with a solution. It’s not uncommon to link to Wiki articles and even so uncommon is the absense of a JavaScript library. Given the static link structure of Wiki articles, it’s not that hard to extract the part that says what article you link to. My solution involves jQuery, some raw JavaScript, some PHP, some configuration using .htaccess and either a subdomain or a folder on a working domain.

So, what’s the big idea?

We’re going to make a proxy for Wikipedia, using the TXT records, which we will use to generate balloons with a summary of the Wiki for that keyword. This way we solve three things:

  1. We don’t bloat our article, but do give extra info for difficult terms;
  2. We provide readers who don’t like to constantly be sent to Wikipedia with a good summary of the article;
  3. People whom still wish to read the full article can click on the link and still get all the info needed;

As an addition, it’s a great support for SEO, because linking to articles with extra information on your article is what helps you rank high on “the Google”.

Provide me with the goodies

Okay, so lets assume you either made a new subdomain or a folder on your website where we send the queries to. First we want to prepare our Apache settings:

RewriteEngine On
RewriteBase /home/wiki/public_html/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

The RewriteBase should fall back to the current working directory by default, but sometimes this works and sometimes this doesn’t, so making this static prevents it from failing. The conditions are there to assure the file we requested doesn’t already exist (which we will use later on to make a cache) and the same for directories. Next up we make sure the request in a whole is given to index.php.

Now we want to populate index.php with some code:

<?php
$q=substr($_SERVER['REQUEST_URI'],1);
exec('dig +short txt '.$q.'.wp.dg.cx',$dig);
$dig = str_replace('" "','',implode("\n",$dig));
if(!file_exists($q) && !empty($dig)) { file_put_contents($q,$dig); }
die($dig);
?>

And what does this do? Well, maybe I should give an example of how we request the text first:

If I want to know something about Unix systems, I simply make a request for http://wiki.johmanx.com/unix.
Since I actually went for making a special subdomain, wiki, I can make requests like that.

Back to the code: In the first line we see the variable $q, which is simply the Request URI minus the first character, since in my example that would be /unix and we only want unix. Then comes a system command. Windows users should actually stop reading at this point, because dig is in no way available on a Windows server. No worries, because when I get to the Ajax part, there’s still a lot of fun stuff we can do. The dig command is a tool in Linux that allows us to get information about DNS records from a domain. In this case we don’t request information about the typical A record, but we do so for the TXT record. Now, normally we use that to store some information that clarifies what we do with a certain domain, but something.wp.dg.cx actually stores the wiki summary for the term something. By giving it the +short parameter, we make sure we don’t get any fancy data for that domain, but only the contents of the TXT record.

The only thing that remains in the code is a basic filtering for weird closing and opening quotes and to make sure that when it ever happens that a TXT record exists of multiple lines, we actually make sure all the lines are present. Then we check if we don’t overwrite a file by putting this in cache (god forbid I don’t do this and someone gets the bright idea to create and query the article index.php). Then finally we make sure to stop the script and end with only echoing the text we just queried.

But now we’re stuck with Cross Domain AJAX calls

Hmm. I have to give you that. Well, the reason I went for a subdomain approach is because I can provide Windows users with my proxy as where the Linux users can put this in a folder and get a more optimal aproach. So, provided we stick with the cross domain solution, we need to make something like XD.php for our AJAX. It should contain something like this:

<?php
echo file_get_contents('http://wiki.johmanx.com/'.$_SERVER['QUERY_STRING']);
?>

Now, if we request XD.php?waffles, we should get some text rambling about a “batter based cake”.
For a simple AJAX call, I tend to fall back to this simple piece of code:

var createXMLHttpRequest = function() {
    if (window.ActiveXObject) {
        return function() { xmlHttp = new ActiveXObject("Microsoft.XMLHTTP"); }
    } 
    else if (window.XMLHttpRequest) {
        return function() { xmlHttp = new XMLHttpRequest(); }
    }
}();
 
function GETRequest(queryString,cb) {
    createXMLHttpRequest();
 
    xmlHttp.onreadystatechange = cb;
    xmlHttp.open("GET", queryString, true);
    xmlHttp.send(null);
}
 
function POSTRequest(url,query,cb) {
    createXMLHttpRequest();
 
    xmlHttp.open("POST", url, true);
    xmlHttp.onreadystatechange = cb;
    xmlHttp.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");    
    xmlHttp.send(query);
}
 
function ajaxDone() {
    if(xmlHttp.readyState == 4 && xmlHttp.status == 200) {
        return true;
    }
    return false;
}

Now we can make a very jQuery like request for our article:

$("body")
	.delegate("a","hover",
		function(){
			var keyword = $(this).html();
			GETRequest('XD.php?'+keyword,
				function(){
					if(ajaxDone()){	
						$('#balloon-keyword-'+keyword).html(xmlHttp.responseText).show();
					}
				}
			);
		}
	)
	.delegate(".balloon","mouseout",
		function(){
			$(this).hide();
		}
	);

Provided each summary is given a link to the original article, we could add a filter to automatically make anchor links, but to keep this article as simple as possible for quick implementation, I provide the filter apart from that bit of jQuery:

function txt2html(txt) {
  var exp = /(\b(https?|ftp|file):\/\/[-A-Z0-9+&@#\/%?=~_|!:,.;]*[-A-Z0-9+&@#\/%=~_|])/ig;
  return txt.replace(exp,"<a href='$1' target='_blank'>$1</a>");
}

That way you could apply a filter to the summary, which will automatically transform a URL to an anchor link.

Now, say you’d want to do this the proper way and put the wiki script in a folder called /wiki/, you could replace ‘XD.php?’ with ‘/wiki/’ and get the same result. This way you don’t have to make Cross Domain calls and you can make optimal use of the cache I provided.

Some other thoughts

Now we’re finished setting up our fine proxy, some thoughts come to mind.

This would make for a good HTTP proxy to apps for the iPhone, iPad and Android platforms. I could see a number of apps coming from this. On top of that list is an App for quickly getting extra information about the ingredients of your recipe. Somewhat lower I’d suggest a Trivia app that generates random questions and gets the answers via a similar proxy. In my opinion, there’s a whole number of apps that could bloom this way. Anyone up for the challenge?

Finding clients the easy way

This is my first post in English, so stay with me during this test as i progress and make some epic fails.

Finding clients the regular way

Normally i wake up, take a shower, have breakfast and check my mail. If i find myself in the position of not having enough work at the moment, i’ll start browsing job boards and search Twitter for relevant information.

The downside to this is the huge amount of time this takes. Not only that, but it involves browsing to different sites with different layouts and different means of filtering and searching for jobs. This can be a tiny pain in the ass!

Job Listings

Finding clients in under 5 minutes

So, provided i browse all these sites and bore myself to death by watching an endless Twitter stream, it could take me hours to come up with a list of potential clients. This just doesn’t cut it. It’s too consuming and too less rewarding.

Hell, i’m a web developer. Why can’t this be more like reading those nice job adverts in the news paper. Honestly? It’s not that hard to accomplish just that. Now, when i wake up i read my mail and browse through job.johmanx.com for some potential victims to pay me some moneyz.

Building a personal job board

Ofcourse i had to go around a really simple way too solve this “problem” of mine, because time is money and all this had to do was display an overview, so here goes.

First i assembled a list of job boards i wanted to follow:

$boards = array(
	'odesk'			=>	'http://www.odesk.com/jobs/rss?c1=Web+Development',
	'37 signals - design'		=>	'http://jobs.37signals.com/categories/1/jobs.rss',
	'37 signals - programming'	=>	'http://jobs.37signals.com/categories/2/jobs.rss',
	'freelanceswitch - design'	=>	'http://feeds.feedburner.com/FSJobsDesign',
	'freelanceswitch - devel'	=>	'http://feeds.feedburner.com/FSJobsProgramming',
	'freelanceswitch - misc'	=>	'http://feeds.feedburner.com/FSJobsMisc',
	'authentic jobs'		=>	'http://www.authenticjobs.com/rss/custom.php?terms=&amp;type=freelance&amp;cats=3,4,2,5,6,'
);

As you may have noticed, i sticked to specific categories with the most value to me.

I also wanted to follow a few search queries on twitter, so i could follow people in need of a developer:

$twitter_searches = array(
	'need web designer',
	'hire web designer',
	'need web developer',
	'hire web developer',
	'hire php'
);

Then comes the “chooka-chook” and we give those feeds some rails to ride on:

/***
 * @function	parse_universal(var $feed, int $retnr)
 * @description	parse a universal XML feed
 * @return	array array(var $feed, var $name)
 */
 
function parse_universal($feed,$name="")
	{
 
		$feed = simplexml_load_file($feed);
 
		$child = $feed->xpath("channel");
		$child = $child[0]->xpath("item");
		$feed = array();
		foreach($child AS $item)
			{
				$title = $item->xpath("title");
				$date = $item->xpath("pubDate");
				$timestamp = strtotime(substr($date[0][0],0));
				$guid = $item->xpath("link");
				$feed[] = array($timestamp,$name,substr($title[0][0],0),$guid[0][0]);
			}
		return $feed;
	}
 
/***
 * @function	socialdate(int $timestamp)
 * @description	Transform time in human readable difference in time
 * @return	var $val
 */
 
function socialdate($timestamp)
	{
		if($timestamp == 0) { return "?"; }
 
		$diff = time()-$timestamp;
 
		switch($diff)
			{
				case ($diff&lt;120):		$val = round($diff,0)." seconds";			break;
				case ($diff&lt;7200):		$val = round($diff/60,0)." minutes";			break;
				case ($diff&lt;48*3600):	$val = round($diff/3600,0)." hours";			break;
				case ($diff&lt;14*24*3600):	$val = round($diff/(3600*24),0)." days";			break;
				case ($diff&lt;60*24*3600):	$val = round($diff/(3600*24*7),0)." weeks";		break;
				case ($diff&lt;730*24*3600):	$val = round($diff/(3600*24*30),0)." months";		break; 				case ($diff>=365*24*3600):	$val = round($diff/(3600*24*365),0)." years";		break;
			}
 
		return '-'.$val;
	}
 
/***
 * @function	web2txt(var $txt)
 * @description	Transform URLs to Anchor links
 * @return	var $txt
 */
 
function web2txt($txt="")
	{
		$txt = eregi_replace('(((f|ht){1}tp://)[-a-zA-Z0-9@:%_+.~#?&amp;//=]+)', '<a href="\1" target="_blank">\1</a>', $txt);
		return $txt;
	}

web2text() is a simple function to look for URLs and transform them to clickable links.
socialdate() is a function to determine the distance between now and then in human readable text.
parse_universal() needs some explenating on my part.

Parsing an XML feed

While parsing an XML document is a fairly routine task, i made it a bit more complex to suit my needs. For instance, i wanted to order everything by date, including all the boards and twitter queries. This can be done simply by using the rsort() function of PHP. However, i’m using a multidimensional array which needs to be sorted in reverse order. By default, rsort tries to order everything as logically possible, so in case of an array with multiple dimensions, the first entry of an array within the array is sorted with the first entry of the next array within the array. Confusing? I’ll just give an example:

$array = array(
  array(4,6,'amber'),
  array(6,4,'mapel'),
  array(2,5,'chocolate')
);
rsort($array);
print_r($array);

The output:

Array
(
    [0] => Array
        (
            [0] => 6
            [1] => 4
            [2] => mapel
        )

    [1] => Array
        (
            [0] => 4
            [1] => 6
            [2] => amber
        )

    [2] => Array
        (
            [0] => 2
            [1] => 5
            [2] => chocolate
        )

)

As you can see and as i told before, the first entry of each array is used to compare and pick the order. In the function parse_universal() we make use of this by putting the UNIX timestamp as first entry in the array.

Next up we pass it a name, so we can later on determine what the source of the message actually was. This is especially handy for finding resources that just clutter your stream. After that we pass on the text of the message and the place of the original message.

Parsing the resources

Now we finally got everything ready to be processed into our own board, let’s get some magic happening:

$feed = array();
foreach($boards AS $board=>$url)
	{
		$board = @parse_universal($url,$board);
		$feed = @array_merge($feed,$board);
	}
sleep(2);
foreach($twitter_searches AS $query)
	{
		$search = @parse_universal('http://search.twitter.com/search.rss?q='.$query,"twitter: ".$query);
		$feed = @array_merge($feed,$search);
	}
rsort($feed);
 
foreach($feed AS $row)
	{
		print '
<div class="rounded">
<a href="'.$row[3].'" target="blank">'.web2txt($row[2]).'</a>
<div class="bottom"><small>'.socialdate($row[0]).'</small> - <small><em>'.$row[1].'</em></small></div>
</div>
';
	}

Unfortunately we need the array_merge() function, because we want to generate multiple arrays at first and there’s no other way to merge an array. Sounds fair enough. It’s just not an optmized way of handling arrays. The reason i put in a different foreach for the twitter functionality is just because i’m lazy and don’t want to copy and paste that feed url everytime i change my search queries. As you can see i pass in both the feed url and a name, which will be used to identify the messages later on.

When the final feed is processed, we use an array with processed data:

$row[(int)]

  • 0: The timestamp which will be transformed using socialdate()
  • 1: The name of the feed from which it originated
  • 2: The text of the current message, checked on possible links using web2txt()
  • 3: The URL of the original location of the message

Now we’re almost there. We already have the following garbage:
Raw job list

Giving it some style

As it is, this list is garbage. You can’t read it easily and it’s really hard to filter anything usefull. Luckily we can give it some spice, using CSS:

	.rounded{
		-webkit-border-radius: 16px;
		-moz-border-radius: 16px;
		border-radius: 16px;
		background:#eee;
		margin:3px;
		padding:8px;
		float:left;
		width:478px;
		height:90px;
		position:relative;
		font-family:Tahoma;
	}
	.bottom{
		position:absolute;
		bottom:8px;
	}

Looky here:

Styled job list

Now, that’s more like it.

Next up: Automating the shizzle

Ofcourse reading N amount of feeds and search queries takes some time. Time you don’t wanna spend waiting, because this removes the whole purpose of saving time using a personal board. Normally i would go with something like this:

*/15 * * * * php /path/to/parse.php > /home/job.johmanx.com/public_html/cache.html

However, this isn’t going to work, because sometimes a feed has an enormous hold up or inconsistent data. I have tried everything to extend my parsing time, but i always end up with a 500 – Internal server error. On this account you have to minimize your resources and build in a failsafe, like i did:

*/15 * * * * php /path/to/cacher.php

Cacher.php:

$content = file_get_contents('/path/to/parse.php');
if(strlen($content)>1200){file_put_contents('/home/job.johmanx.com/public_html/cache.html',$content);}

This way you don’t get an empty page when it failed, which can be really frustrating.

Index.php:

<!doctype html> 
<html> 
	<head> 
		<title>Job listings | Joh Man X - Clear webdevelopment</title> 
		<link rel="stylesheet" href="stylesheet/style.css" type="text/css" /> 
 
		<!--[if IE]>
			<script src="http://html5shiv.googlecode.com/svn/trunk/html5.js"></script>
		<![endif]-->
 
	</head> 
 
	<body> 
 
		<div id="pagewrap"> 
 
			<section> 
 
				<div class="wrapper">
					<?php print file_get_contents('cache.html'); ?>
					<br class="clear" /> 
				</div> 
 
			</section>
 
		</div> 
	</body> 
</html>

The final product

And there you have it. After some minor alterations, i had my job board.

job.johmanx.com