Here’s a way to make them do something useful (and maybe even cool) : AutoHotkey. AutoHotkey is a free macro utility that lets you create scripted hotkeys for any key or key combination. With this utility, you can make the previously useless keys launch your favorite applications, control your media player, simplify copy/pasting, and more. I’ve included a small selection of handy hotkey scripts below, but you can find thousands more in the Script Showcase and the AutoHotkey forums.

Okay, lets move on to the actual scripts.

Hot Hotkeys

Show desktop

As you might know, you can hide/restore all open windows by pressing the Windows key and the D key together. With AutoHotkey, you can assign the same function to a single key – say, CapsLock. Here’s the AHK script :

CapsLock::#d

Open or focus the browser

If you’re like me, you probably tend Alt-Tab to the web browser every five minutes to run a Google search, read some online documentation or check for new emails. This can be simplified by creating a hotkey that, when pressed, brings the browser window to the front (or launches the browser if it isn’t already running). This script by Niels Teglsbo does just that :

; Function to run a program or activate an already running instance
RunOrActivateProgram(Program, WorkingDir="", WindowSize=""){
    SplitPath Program, ExeFile
    Process, Exist, %ExeFile%
    PID = %ErrorLevel%
    if (PID = 0) {
        Run, %Program%, %WorkingDir%, %WindowSize%
    }else{
        WinActivate, ahk_pid %PID%
    }
}
; Make CapsLock run or activate Internet Explorer
CapsLock::RunOrActivateProgram("c:\Program Files\Internet Explorer\iexplore.exe")

The example above uses Internet Explorer, but you can easily substitute it with your own favourite browser by editing the file path.

Disable CapsLock

Although CapsLock can be modified to become a useful hotkey, some people would rather just get rid of it because it’s too easy to press it accidentally. Well, all it takes is a single line script :

SetCapsLockState, AlwaysOff

Alternatively, you could remap CapsLock to act as Ctrl. This way it can both be useful and still remain harmless when accidentally pressed :

CapsLock::Ctrl

Single-key Cut/Copy/Paste

This script will make the right-side Alt, Windows and Menu keys function like Cut, Copy and Paste :

RAlt::^x 	; Right Alt as Ctrl-X (Cut)
RWin::^c 	; Right Windows key as Ctrl-C (Copy)
AppsKey::^v	; Menu/Apps key as Ctrl-V (Paste)

You can find an alternative setup that uses only the CapsLock key in this StackOverflow thread.

Incrementally switch windows

iSwitchw lets you incrementally switch between open windows using CapsLock (though you could easily modify it to use a different key).

It’s a window switcher which shows the matching window titles as you type in your search string incrementally. If only one window is left it is activated immediately (configurable). Otherwise, you can type in more characters (or delete some with backspace) or select between the matching windows using cursor up/down/enter, or you can cancel the window with Esc.

You can use any substring of any window. For example, if you want to switch to word then you can type rd and there is a good chance word is selected immediately. Or type “notepad” and select quickly between the notepad windows with the cursor.

The script itself is too long to be included here. You can find it in the iSwitchw forum thread.

Control your media player

If your keyboard doesn’t have media keys, you can use AutoHotkey to remap some of your unused keys to act as passable substitutes. For example, I use the block of three keys to the right of the Space bar (on  a 104-key keyboard) to control Winamp :

RWin::
IfWinNotExist ahk_class Winamp v1.x
    return
ControlSend, ahk_parent, c  ; Pause/Unpause
return

RAlt::
IfWinNotExist ahk_class Winamp v1.x
    return
ControlSend, ahk_parent, z  ; Previous track
return

AppsKey::
IfWinNotExist ahk_class Winamp v1.x
	return
ControlSend, ahk_parent, b  ; Next track
return

Easy window dragging

Normally, a window can only be dragged by clicking on its title bar. Easy Window Drag extends that so that any point inside a window can be dragged. Just hold down CapsLock or the middle mouse button while clicking, then drag the window to a new position.

You can download the script here.

Other Uses For Unused Keys

In addition to AutoHotkey, there are several other applications that can take advantage of typically unused keys.

Enso Launcher

Enso Launcher is yet another cousin of Quicksilver/Launchy. It turns CapsLock into a multifunctional hotkey that lets you enter commands, launch applications, manipulate windows, access your bookmarks, quickly perform simple calculations, and more.

Network Lights

Finally, here’s a little bonus for all who love to watch the blinkenlights : there’s a small freeware utility called Network Lights that blinks the ScrollLock and NumLock LEDs indicating network traffic. It may not be very useful per se, but the thought of turning the keyboard into a network monitor of sorts certainly possesses a notable coolness factor.

Image credit : staarpoint @ sxc.hu

For non-programmers : If you want something simple and don’t really care about detecting sites that hide their version info, check out the Show WordPress Version Greasemonkey script. Whenever you visit a WordPress blog, this simple userscsript will pop up  a small box showing the WP version.

Introduction

So how does one go about detecting the WordPress version? Simple – every WordPress site contains certain WordPress-specific footprints. A “footprint” can be anything from a unique piece of HTML code present on the front page to a specific subdirectory or file that’s only found on WordPress-based sites. Some sites try to hide the WordPress footprints, but most still leave something visible. And more often than not, this something is enough to determine that the site is running WordPress, and even find out the version number.

Lets see what these footprints are, shall we?

The generator META Tag

This is probably the most well-known way to detect the WordPress version. By default, WP will add the following META tag to every page of the site :

<meta name="generator" content="WordPress 2.8.6" />

All you need to do is check for the presence of this tag and you’ll immediately know that the site is running WordPress. Even the version number is helpfully included. However, this is also the least reliable technique, as it’s very easy to replace the version number with a fake one or remove the tag altogether.

The RSS Feed generator Tag

Even if the meta tag is gone, there’s still another place where the version number might be on display – the RSS feed. The WordPress generated feed is usually available at example.com/?feed=rss2 and contains a tag that looks like this :

<generator>http://wordpress.org/?v=2.8.</generator>

As you can see, the version number is right there.

CSS version data

Another place where WP discloses version information is the /wp-admin/upgrade.php page. This page uses an external CSS stylesheet, and the stylesheet URL includes a “ver”/”version” parameter. Here’s an example from WP 2.8.6 :

<link rel='stylesheet' id='install-css'  href='http://example.com/wp-admin/css/install.css?ver=20090514' type='text/css' media='all' />

This footprint is a (little) bit trickier to parse that the first two. Up to WordPress 2.5.1, the stylesheet URL contained the actual WordPress version in the “version” parameter. In later releases, it was replaced with a “ver” parameter containing some kind of a date; probably indicating the last time that CSS file was modified. I’ve found that the different date corresponds to each major WP release :

• 20090514 – 2.8.*
• 20081210 – 2.7.*
• 20080708 – 2.6.*

In some ways, this is actually better for our purposes than having the full version number in clear view. As I mentioned before, the version number can be easily spoofed. The date, however, can’t. I haven’t seen any plugins that would let you change or remove this footprint.

If /wp-admin/upgrade.php happens to be inaccessible on some sites, you can also find a similar footprint in /wp-includes/js/tinymce/wp-mce-help.php.

Version-specific files

Finally, almost every single new release of WordPress includes one or more files that weren’t there in the previous version. Many of these files – like images and JS scripts – are accessible to normal visitors. This means you can determine the WP version simply by checking for the presence of specific files.

I’ve compiled a small list of the last 6 major WP releases + a new web-accessible file that was added in each release :

• 2.2 – /wp-includes/images/rss.png
• 2.3 – /wp-includes/js/scriptaculous/sound.js
• 2.5 – /wp-includes/js/jquery/suggest.js
• 2.6 – /wp-includes/images/blank.gif
• 2.7 – /wp-includes/js/comment-reply.js
• 2.8 – /wp-includes/js/codepress/codepress.js

This list is by no means comprehensive, but you can easily make your own by downloading older versions of WP and comparing them.

Happy data mining

What’s a Parked Domain?

Broadly speaking, a parked domain is a website that contains no real content. Parked domains can have a benign purpose, like when someone puts a “Coming Soon” message on a domain reserved for an in-development site. However, much more commonly they are nothing but single-page “portals” stuffed with ads, bereft of any value to the unfortunate visitor. Typosquatters also frequently park their domains.

How To Detect Parked Sites

Let me state up-front that there is no single fool-proof method to detect parked domains. Each domain parking service has a lightly different page template and site structure, so even if you build a tool that can identify domains parked with SEDO.com with 100% accuracy it might still be clueless about sites parked with GoDaddy. The best approach is to use a combination of different detection algorithms.

Here’s a summary of techniques that I found promising.

Check for subdomain wildcarding and ubiquitous redirects

A lot of parked domains will gladly accept any HTTP request you throw at them, even if you try to access a non-existent subdomain or page. Instead of the typical “Not Found” error, a parked domain will either return a dynamically generated page full of ads or redirect you to the front page.  So you can check if a domain is parked is by requesting a bogus subdomain (e.g. http://bogus.example.com/) or page (e.g. http://example.com/this-page-doesnt-exist.html).

Another sign that a domain might be parked is that it redirects to a subdomain or a sub-directory on a different domain. For example, some domain parking services redirect parked domains to URLs like http://domain-name.domainparking.com or http://domainparking.com/?domain=domain-name.

There is sometimes a good reason to redirect a domain, like when the site has moved to a new address. How can we distinguish between legitimate redirects and parked sites? Legitimate redirects usually use (or should use) the 301 response code, while parked domains use the 302 code.

There are also lots of parked domains that don’t use redirects at all, so this method is unfortunately rather prone to false positives and negatives.

Examine the WHOIS information

The WHOIS info of a parked domain usually points to the domain parking service that owns/controls the domain. You can use this to build a database of known-suspicious WHOIS records that can be used to identify parked domains. Alternatively, if you have access to a big WHOIS database, you could probably catch a good portion of parked sites by flagging any domain registrant that owns a lot of domains (say, more than a hundred).

Use an existing blacklist

Surprise, surprise : there is actually a parked domain blacklist. Run by the schizophrenically named company “I’ve Got a Fang”, the Parked Domains Project is an attempt to build and maintain an extensive database of parked domains. It even has a Firefox extension.

As far as I can tell there is no documented public API, but you can probably reverse-engineer the FF add-on and see what it uses (or just screen-scrape the database if you’re feeling naughty).

Analyse the content

Personally, I think the most promising approach for detecting parked domains automatically is content analysis. Parked domains all have something in common – the ads. If you can come up with a reliable and automated way to detect if a page is devoted solely to advertising, you will be able to identify parked domains pretty accurately.

The paper Content-Based Approach for Identifying Textual Ads-Portal Domains [PDF] describes a very effective algorithm for detecting ads-only domains :

In this paper, we develop a machine-learning-based classifier to identify ads-portal domains. The features of the classifier are extracted from the web content of the domain. In developing the classifier, we first create negative and positive samples set. Then, we identify set of features that are effective in distinguishing ads-portal domains. Finally, we combine these features along with other keyword-based features into a machine learning classifier. The resulting classifier has 97% accuracy in identifying ads-portal domains.

On a related note, my content extraction script could also be used for detecting is a site is low on actual content. Its algorithm is much simpler than the one proposed in the above paper, but – unlike the paper – the source code of my PHP script is freely available

Illustration credit :  clix @ sxc.hu

There are some (though very few they may be) edge case where regular expressions might work better, so I will discuss both approaches in this post.

Extracting Tags With DOM

PHP 5 comes with a usable DOM API built-in that you can use to parse and manipulate (X)HTML documents. For example, here’s how you could use it to extract all link URLs from a HTML file :

//Load the HTML page
$html = file_get_contents('page.htm');
//Create a new DOM document
$dom = new DOMDocument;
 
//Parse the HTML. The @ is used to suppress any parsing errors
//that will be thrown if the $html string isn't valid XHTML.
@$dom->loadHTML($html);
 
//Get all links. You could also use any other tag name here,
//like 'img' or 'table', to extract other tags.
$links = $dom->getElementsByTagName('a');
 
//Iterate over the extracted links and display their URLs
foreach ($links as $link){
	//Extract and show the "href" attribute. 
	echo $link->getAttribute('href'), '<br>';
}

In addition to getElementsByTagName() you can also use $dom->getElementById() to find tags with a specific id. For more complex tasks, like extracting deeply nested tags, XPath is probably the way to go. For example, to find all list items with the class “foo” containing links with the class “bar” and display the link URLs :

//Load the HTML page
$html = file_get_contents('page.htm');
//Parse it. Here we use loadHTML as a static method
//to parse the HTML and create the DOM object in one go.
@$dom = DOMDocument::loadHTML($html);
 
//Init the XPath object
$xpath = new DOMXpath($dom);
 
//Query the DOM
$links = $xpath->query( '//li[contains(@class, "foo")]//a[@class = "bar"]' );
 
//Display the results as in the previous example
foreach($links as $link){
	echo $link->getAttribute('href'), '<br>';
}

For more information about DOM and XPath see these resources :

• DOM API documentation
• W3Schools XPath tutorial
• XPath language specification

Honourable mention : Simple HTML DOM Parser is a popular alternative HTML parser for PHP 5 that lets you manipulate HTML pages with jQuery-like ease. However, I personally wouldn’t recommend using it if you care about your script’s performance, as in my tests Simple HTML DOM was about 30 times slower than DOMDocument.

Extracting Tags & Attributes With Regular Expressions

There are only two advantages to processing HTML with regular expressions – availability and edge-case performance. While most parsers require PHP 5 or later, regular expressions are available pretty much anywhere. Also, they are a little bit faster than real parsers when you need to extract something from a very large document (on the order of 400 KB or more). Still, in most cases you’re better off using the PHP DOM extension or even Simple HTML DOM, not messing with convoluted regexps.

That said, here’s a PHP function that can extract any HTML tags and their attributes from a given string :

/**
 * extract_tags()
 * Extract specific HTML tags and their attributes from a string.
 *
 * You can either specify one tag, an array of tag names, or a regular expression that matches the tag name(s). 
 * If multiple tags are specified you must also set the $selfclosing parameter and it must be the same for 
 * all specified tags (so you can't extract both normal and self-closing tags in one go).
 * 
 * The function returns a numerically indexed array of extracted tags. Each entry is an associative array
 * with these keys :
 * 	tag_name	- the name of the extracted tag, e.g. "a" or "img".
 *	offset		- the numberic offset of the first character of the tag within the HTML source.
 *	contents	- the inner HTML of the tag. This is always empty for self-closing tags.
 *	attributes	- a name -> value array of the tag's attributes, or an empty array if the tag has none.
 *	full_tag	- the entire matched tag, e.g. '<a href="http://example.com">example.com</a>'. This key 
 *		          will only be present if you set $return_the_entire_tag to true.	   
 *
 * @param string $html The HTML code to search for tags.
 * @param string|array $tag The tag(s) to extract.							 
 * @param bool $selfclosing	Whether the tag is self-closing or not. Setting it to null will force the script to try and make an educated guess. 
 * @param bool $return_the_entire_tag Return the entire matched tag in 'full_tag' key of the results array.  
 * @param string $charset The character set of the HTML code. Defaults to ISO-8859-1.
 *
 * @return array An array of extracted tags, or an empty array if no matching tags were found. 
 */
function extract_tags( $html, $tag, $selfclosing = null, $return_the_entire_tag = false, $charset = 'ISO-8859-1' ){
 
	if ( is_array($tag) ){
		$tag = implode('|', $tag);
	}
 
	//If the user didn't specify if $tag is a self-closing tag we try to auto-detect it
	//by checking against a list of known self-closing tags.
	$selfclosing_tags = array( 'area', 'base', 'basefont', 'br', 'hr', 'input', 'img', 'link', 'meta', 'col', 'param' );
	if ( is_null($selfclosing) ){
		$selfclosing = in_array( $tag, $selfclosing_tags );
	}
 
	//The regexp is different for normal and self-closing tags because I can't figure out 
	//how to make a sufficiently robust unified one.
	if ( $selfclosing ){
		$tag_pattern = 
			'@<(?P<tag>'.$tag.')			# <tag
			(?P<attributes>\s[^>]+)?		# attributes, if any
			\s*/?>					# /> or just >, being lenient here 
			@xsi';
	} else {
		$tag_pattern = 
			'@<(?P<tag>'.$tag.')			# <tag
			(?P<attributes>\s[^>]+)?		# attributes, if any
			\s*>					# >
			(?P<contents>.*?)			# tag contents
			</(?P=tag)>				# the closing </tag>
			@xsi';
	}
 
	$attribute_pattern = 
		'@
		(?P<name>\w+)							# attribute name
		\s*=\s*
		(
			(?P<quote>[\"\'])(?P<value_quoted>.*?)(?P=quote)	# a quoted value
			|							# or
			(?P<value_unquoted>[^\s"\']+?)(?:\s+|$)			# an unquoted value (terminated by whitespace or EOF) 
		)
		@xsi';
 
	//Find all tags 
	if ( !preg_match_all($tag_pattern, $html, $matches, PREG_SET_ORDER | PREG_OFFSET_CAPTURE ) ){
		//Return an empty array if we didn't find anything
		return array();
	}
 
	$tags = array();
	foreach ($matches as $match){
 
		//Parse tag attributes, if any
		$attributes = array();
		if ( !empty($match['attributes'][0]) ){ 
 
			if ( preg_match_all( $attribute_pattern, $match['attributes'][0], $attribute_data, PREG_SET_ORDER ) ){
				//Turn the attribute data into a name->value array
				foreach($attribute_data as $attr){
					if( !empty($attr['value_quoted']) ){
						$value = $attr['value_quoted'];
					} else if( !empty($attr['value_unquoted']) ){
						$value = $attr['value_unquoted'];
					} else {
						$value = '';
					}
 
					//Passing the value through html_entity_decode is handy when you want
					//to extract link URLs or something like that. You might want to remove
					//or modify this call if it doesn't fit your situation.
					$value = html_entity_decode( $value, ENT_QUOTES, $charset );
 
					$attributes[$attr['name']] = $value;
				}
			}
 
		}
 
		$tag = array(
			'tag_name' => $match['tag'][0],
			'offset' => $match[0][1], 
			'contents' => !empty($match['contents'])?$match['contents'][0]:'', //empty for self-closing tags
			'attributes' => $attributes, 
		);
		if ( $return_the_entire_tag ){
			$tag['full_tag'] = $match[0][0]; 			
		}
 
		$tags[] = $tag;
	}
 
	return $tags;
}

Usage examples

Extract all links and output their URLs :

$html = file_get_contents( 'example.html' );
$nodes = extract_tags( $html, 'a' );
foreach($nodes as $link){
	echo $link['attributes']['href'] , '<br>';
}

Extract all heading tags and output their text :

$nodes = extract_tags( $html, 'h\d+', false );
foreach($nodes as $node){
	echo strip_tags($link['contents']) , '<br>';
}

Extract meta tags :

$nodes = extract_tags( $html, 'meta' );

Extract bold and italicised text fragments :

$nodes = extract_tags( $html, array('b', 'strong', 'em', 'i') );
foreach($nodes as $node){
	echo strip_tags( $node['contents'] ), '<br>';
}

The function is pretty well documented, so check the source if anything is unclear. Of course, you can also leave a comment if you have any further questions or feedback.

Side-note : No, I haven’t been completely consumed by an inexorable desire to build fun graphical toys instead of practically useful software, why do you ask? In fact, there’ll be some WordPress-related announcements later this week…

Learning To Fly Fall

While I can’t claim intimate familiarity with the algorithms the aforementioned games use to simulate fire or explosions, it’s pretty easy to come up with a way to implement falling sand/flowing water. First, you need to figure out under what conditions something can fall/flow downwards. For a basic simulation this could be just a small set of very simple rules :

• For each entity that is affected by gravity
  • If there’s an empty space below it, move it down.
  • If there’s an empty space down and to the left, move it down and to the left.
  • If there’s an empty space down and to the right, move it down and to the right.

Next, you need to decide how to represent the simulation world and all the objects (like immovable walls, moving water particles et cetera) that it contains. One of the most common solutions is to use a two-dimensional array where each array element is an numeric ID of the material contained in that map tile. For example, 0 = air (or an empty square), 1 = wall, 2 = water, and so on.

In most falling sand games each array element would correspond to a single pixel on the game’s screen. However, in my example application I used 20×20 pixel tiles instead to make the underlying simulation behaviour easier to see.

If possible, use byte or char as the array element datatype. Using int’s might seem more straightforward, but it would also take up at least four times more memory and hurt performance. It’s also a good idea to make the array just a bit bigger than the actual map dimensions and create a single-tile “buffer zone” around the map so that you can avoid checking for boundary conditions during the simulation.

Finally, during the simulation step we just loop over the entire array and apply the simple rules I mentioned above to each tile that contains material that should be affected by gravity.

Getting It Right

If you go ahead and implement the above algorithm as written, you will get a simulation that kind-of works, but quickly grinds to a halt on big maps and sometimes behaves in unexpected ways. For example, you might see sand tiles that seem to “teleport” to the bottom of the map instead of falling gradually and water that always flows left after hitting a free-standing ground tile (even though there’s a perfectly suitable channel available on the right side). Lets see what we can do about that.

Keeping track of updated tiles

The “teleportation” behaviour can happen if our simulation algorithm doesn’t keep track of which tiles it has already moved around during the current simulation step. For example, first it loops over the top row of the map, finds a water tile A and moves it downwards one row. Then it loops over the second row, finds the tile A, moves it again, goes to the third row, moves the same water tile again… and so on, until the water tile A hits a solid tile or falls off the map.

To prevent this bug we need to somehow remember which tiles have been updated already. Since I wanted to keep the memory use low, I used the most-significant bit of each map array element to indicate whether that tile was last updated on an even-numbered simulation step (0) or an odd-numbered step (1). Then, at the start of each step, the algorithm calculates what the flag should be for tiles updated in that step :

update_mask = byte ( ( simulationFrameCount & (long)1 ) << 7 );
//this basically takes the modulo of frame-count / 2 and shifts the result 7 places to the left

…then sets the msb of each tile it updates to that value and automatically skips tiles that have it already set to that value.

Pseudo-random movement

When a falling particle hits a free-standing fixed tile on a 2D map it has two choices – it can either move down and left, or down and right. In the naive algorithm I discussed above it will always try to move left first, leading to weirdness. Obviously we want it to choose at random, but on a big map generating a new random number for each tile would be slow. Instead, lets use the same approach as for the “teleportation” problem and move it either diagonally left if the current simulation step is even-numbered and or diagonally right if it’s odd-numbered. This can still introduce some artifacts in the simulation behaviour, but they’re rarer and and significantly less noticeable.

Optimization

Speaking of optimization, we don’t really need to update each gravity-enabled tile every time. Many of them will eventually end up in a situation where they can’t move anywhere (e.g. water at the bottom of a puddle or sand resting on a flat surface), so it would be nice if we could quickly skip them and not waste our time trying to apply the simulation rules to these static tiles. This would help performance quite a bit.

The way I implemented is with the help with another bit-flag. If the simulation algorithm discovers that it can’t move a tile in any direction, it will set the tile’s “static” bit-flag. It will also skip any tiles that have the “static” bit-flag set.

There’s some additional book-keeping we need to do to make this work as intended. When an empty space is created somewhere in the map (e.g. by a water tile moving down) we need to re-activate any gravity-enabled tiles that are directly above the newly created hole, so that they can fall/flow down into that space.

Possible Improvements

The result is a pretty basic and plain simulation that is useful to explain the basic principle, but it wouldn’t really qualify as a full game. Here are some ideas you could use to improve it :

• Make water flow horizontally if there’s another water tile on top of it. Consider adding a simple pressure model [link to my prev. article].
• Implement fluid density. This one is easy – if the current tile is more dense than the tile below it, swap them.
• Add tile generators, e.g. a faucet that creates a water tile below itself on each simulation step (included in my demo application).
• To improve performance, use a quadtree to keep track of which regions of the map have interacting tiles and quickly skip inactive regions.
• Explore the source code of existing games and look for interesting ideas. Some open-source examples : wxSand, SDL Sand.

Live Demo & Source

Like my previous fluid simulation app, this one was also written in Processing. In addition to a working implementation of the algorithm discussed above it also lets you draw and erase water/ground/faucet tiles using your mouse, pan around the map with the WASD keys, zoom in/out with the mouse wheel and more. However, be warned that the map rendering code is not very optimized so the simulation will degenerate into a slideshow if you zoom all the way out, regardless of how fast your system is.

• Live demo  (requires Java)
• Source code

Big Words, or The Theoretical Part

Fluid dynamics is a complex topic. If you so much as look it up in Wikipedia, you will be immediately assaulted by integrals, differentials and other agents of mathematical insanity. To accurately model a fluid you would need a quite an in-depth understanding of physics and calculus. And while that’s certainly not something one couldn’t figure out given enough time, “proper” fluid simulation is still extremely rare in non-academic applications and games because it’s slow.

One way to create a faster (though less accurate) simulation is by using cellular automata to represent the water. A cellular automaton is basically a grid where each cell can be in one of a finite number of states, plus a set of rules that determine how a cell can change from one state to another. The rules are typically local – that is, they consider only the current cell and it’s direct neighbours when determining the new state. During each step of the simulation, you simply loop through the entire grid and apply the rules to each cell.

When simulating water or other fluids it can more appropriate to store a continuous value (e.g. water mass) in each cell, instead of using discrete states.

CA-Based Water Simulation

Before we go into details, lets take a look at the promised online demo. It will give you a good idea about what you can achieve with this approach :

(klicken Sie auf das Bild; Java required)

Alright, on to the code stuff. Lets try simulating the most common fluid – water.

The basic data structure

//Map dimensions
final int map_width = 16;
final int map_height = 16;
 
//Block types
final int AIR = 0;
final int GROUND = 1;
final int WATER = 2;
 
//Data structures
int[][] blocks = new int[map_width+2][map_height+2];
 
float[][] mass = new float[map_width+2][map_height+2],
          new_mass = new float[map_width+2][map_height+2];

I use two two-dimensional arrays to represent the simulation world. The blocks array defines a basic “map” where each cell can contain ground (solid, stops water flow), air (an empty cell, water can flow in) or water. Each water cell also has a corresponding entry in the mass array that defines how much water it contains. All water cells/blocks start out with one unit of water.

There’s also a third array – new_mass that is used to store intermediate mass values when running the simulation. Operating directly on the mass array is unadvisable because you would get various sideffects like water spreading at different speeds depending on the order in which you update the cells.

Overall algorithm

The main idea is to treat water as a slightly compressible liquid, as described in this article. In terms of implementation, this means that if there are two or more water cells stacked vertically, the bottom cells will be able to hold slightly more water than normal. This way we don’t need to explicitly track pressure to make the water level equalize in communicating vessels – we can just look at how much excess water a cell has, and move it upwards if required.

Lets set up the exact fluid properties as constants :

//Water properties
final float MaxMass = 1.0; //The normal, un-pressurized mass of a full water cell
final float MaxCompress = 0.02; //How much excess water a cell can store, compared to the cell above it
final float MinMass = 0.0001;  //Ignore cells that are almost dry

For example, if one cell contains 1.0 units of water, the cell below it should contain up to 1.02 units, the cell below that one should contain 1.04, the next one 1.06, and so on. There is one however special case that we need to consider to get a plausible simulation – what if the if the top cell contains less than MaxMass units of water? In my implementation the bottom cell will contain a proportionally smaller excess amount, not the usual mass_of_the_cell_above + MaxCompression units.

Based on the above, we can simulate water movement by looping over the mass array and applying these cellular automaton rules to each cell :

1. Take the mass of the current cell and the cell below it and figure out how much water the bottom cell should contain. If it has less than that, remove the corresponding amount from the current cell and add it to the bottom cell.
2. Check the cell to the left of this one. If it has less water, move over enough water to make both cells contain the same amount.
3. Do the same thing for the right neighbour.
4. Do the same thing as in step 1., but for the cell above the current one.

While doing this you also need to keep track of how much water the current cell still has remaining, or you could end up with negative-mass water blocks.

After the new mass of each cell has been calculated, you can safely copy over the new values to the mass array. You also need to update the blocks array so that cells that are now dry get marked as empty and vice versa.

Here’s the simulation code :

void simulate_compression(){
  float Flow = 0;
  float remaining_mass;
 
  //Calculate and apply flow for each block
  for (int x = 1; x <= map_width; x++){
     for(int y = 1; y <= map_height; y++){
       //Skip inert ground blocks
       if ( blocks[x][y] == GROUND) continue;
 
       //Custom push-only flow
       Flow = 0;
       remaining_mass = mass[x][y];
       if ( remaining_mass <= 0 ) continue;
 
       //The block below this one
       if ( (blocks[x][y-1] != GROUND) ){
         Flow = get_stable_state_b( remaining_mass + mass[x][y-1] ) - mass[x][y-1];
         if ( Flow > MinFlow ){
           Flow *= 0.5; //leads to smoother flow
         }
         Flow = constrain( Flow, 0, min(MaxSpeed, remaining_mass) );
 
         new_mass[x][y] -= Flow;
         new_mass[x][y-1] += Flow;   
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Left
       if ( blocks[x-1][y] != GROUND ){
         //Equalize the amount of water in this block and it's neighbour
         Flow = (mass[x][y] - mass[x-1][y])/4;
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain(Flow, 0, remaining_mass);
 
         new_mass[x][y] -= Flow;
         new_mass[x-1][y] += Flow;
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Right
       if ( blocks[x+1][y] != GROUND ){
         //Equalize the amount of water in this block and it's neighbour
         Flow = (mass[x][y] - mass[x+1][y])/4;
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain(Flow, 0, remaining_mass);
 
         new_mass[x][y] -= Flow;
         new_mass[x+1][y] += Flow;
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Up. Only compressed water flows upwards.
       if ( blocks[x][y+1] != GROUND ){
         Flow = remaining_mass - get_stable_state_b( remaining_mass + mass[x][y+1] );
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain( Flow, 0, min(MaxSpeed, remaining_mass) );
 
         new_mass[x][y] -= Flow;
         new_mass[x][y+1] += Flow;   
         remaining_mass -= Flow;
       }
 
 
     }
  }
 
  //Copy the new mass values to the mass array
  for (int x = 0; x < map_width + 2; x++){
    for (int y = 0; y < map_height + 2; y++){
      mass[x][y] = new_mass[x][y];
    }
  }
 
  for (int x = 1; x <= map_width; x++){
     for(int y = 1; y <= map_height; y++){
       //Skip ground blocks
       if(blocks[x][y] == GROUND) continue;
       //Flag/unflag water blocks
       if (mass[x][y] > MinMass){
         blocks[x][y] = WATER;
       } else {
         blocks[x][y] = AIR;
       }
     }
  }
 
  //Remove any water that has left the map
  for (int x =0; x < map_width+2; x++){
    mass[x][0] = 0;
    mass[x][map_height+1] = 0;
  }
  for (int y = 1; y < map_height+1; y++){
    mass[0][y] = 0;
    mass[map_width+1][y] = 0;
  }
 
}

And here’s the function that calculates how to distribute a given amount of water between two vertically adjacent cells :

//Returns the amount of water that should be in the bottom cell.
float get_stable_state_b ( float total_mass ){
  if ( total_mass <= 1 ){
    return 1;
  } else if ( total_mass < 2*MaxMass + MaxCompress ){
    return (MaxMass*MaxMass + total_mass*MaxCompress)/(MaxMass + MaxCompress);
  } else {
    return (total_mass + MaxCompress)/2;
  }
}

Full source code

• Download the source (23 KB)
• You will need the Java-based Processing programming language to run the code.

I didn’t discuss how to render the water, or how to handle user interaction, but the source code also includes a simplistic implementation of those things.

Conclusion

CA-based algorithms have both advantages and disadvantages. On the one hand, they’re relatively fast and the simulation quality is often good enough for games and software toys. On the other, they can produce behaviour that is very unrealistic – for example, water falling into a basin will form a little “hill” and spread out slowly, instead of near-instantly as real water would. Whether this is an acceptable downside depends on your application.

For a different take on simulating fluid dynamics with cellular automata, check out this interview with the author of Dwarf Fortress.

If you want make an unruly application remember its window size, check out AutoSizer. It’s a freeware tool that can keep any program at a specific size and/or position. In addition to the ability to set exact window dimensions and coordinates, you can also configure it to maximize or minimize the application, keep it on top of other windows, or even automatically center it on the desktop.

AutoSizer is also useful for dealing with extra-glitchy apps that crash every couple hours and lose their window settings as a result. I’m looking at you, TweetDeck 0.25.1b.

Windows API includes a subset of functions that provide various performance-related information, which includes data on how busy individual processors or cores are. In this post I’ll show you how to retrieve and use this information. I’ve also included a simple example application that demonstrates the functionality (the download link is at the end of the post) :

Screenshot of the example application

The example app was written in Delphi, but the salient parts are almost purely Windows API so they should be easy to translate to other programming languages.

Algorithm Overview

Here’s a general overview of how to use the performance counter API to get the per-core usage numbers.

Initialization

1. Create a performance query using PdhOpenQuery.
2. Generate a list of performance counter paths (one for each CPU or core) by feeding a wildcard path to PdhExpandWildCardPath. In this case, this is the path that we need :
   \Processor(*/*#*)\% Processor Time
   You can read more about counter path syntax here.

   Note : Unfortunately, the API makes no distinction between multiple physical processors and multiple cores, so you will get a “…\Processor(X)\…” entry for each core.

3. Add the generated counter paths to the query using PdhAddCounter and save the returned handle(s) for later.

Collecting & Displaying Data
CPU usage is typically measured over an interval of time, so you will need to collect at least two data samples at different times to get a meaningful result. Depending on your needs, you could either :

• Collect one sample, sleep() for a second, collect another sample and calculate the result; OR
• Set up a timer or a loop to calculate & displays the CPU usage periodically. That’s what I’ll do in this tutorial.

Regardless of whether you only need a one-time measurement or periodic updates, the process is very similar API-wise. First, call PdhCollectQueryData to update all counters associated with the query. Then call PdhGetFormattedCounterValue with each counter handle to get the actual data. If you set the output format flag to PDH_FMT_DOUBLE the function will give you the CPU/core load percentage as a floating-point value in the range of [0..100].

If you need periodic updates just call these two functions repeatedly.

Cleanup
When you’re done, call PdhCloseQuery to close counters associated with the query and free up allocated system resources. You don’t need to explicitly remove individual counters.

Source

This is the source code of a simple Delphi 2009 application that displays the load percentage of each processor core in a progress bar. The output is updated every second by using a timer component.

unit CoreUsage;
 
interface
 
uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, JwaWindows, ExtCtrls, ComCtrls, Gauges;
 
type
  TForm1 = class(TForm)
    Timer1: TTimer;
    procedure FormCreate(Sender: TObject);
    procedure Timer1Timer(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;
 
  TCounter = record
   Path : string;
   Handle : cardinal;
   mLabel : TLabel;
   mGauge : TGauge;
  end;
 
var
  Form1: TForm1;
  Counters : array of TCounter;
  Query : Cardinal = 0;
 
implementation
 
{$R *.dfm}
 
procedure TForm1.FormCreate(Sender: TObject);
var
 dwSize, h : cardinal;
 cnt, i : Integer;
 pPaths:PWideChar;
 pIterator:PwideChar;
 InstanceId, CounterPath : string;
 status : PDH_STATUS;
begin
  //Create a performance query
  if  PdhOpenQuery(nil, 0, Query) = ERROR_SUCCESS then begin
 
    //Get all valid processor/core usage counter paths by expanding
    //a wildcard path. To do this, we must first call PdhExpandWildCardPath
    //with buffer size set to zero to get the actual required buffer size.
    dwSize := 0;
    pPaths := nil;
 
    status := PdhExpandWildCardPath(
      nil,                                  //search the local computer
      '\Processor(*/*#*)\% Processor Time', //we want CPU usage counters for all CPUs/cores
      pPaths,                               //user-allocated buffer; currently null
      dwSize,                               //buffer size
      0) ;                                  //no flags
 
    if status = PDH_MORE_DATA then begin
      dwSize := dwSize + 1; //+1 byte required in XP and below.
      pPaths := GetMemory(dwSize); //Allocate an output buffer.
 
      //Really get the counter paths.
      status := PdhExpandWildCardPath(
        nil,
        '\Processor(*/*#*)\% Processor Time',
        pPaths,
        dwSize,
        0) ;
 
      if status = ERROR_SUCCESS then begin
 
        cnt := 0;
        SetLength(Counters, 32);
 
        //PdhExpandWildCardPath returns the counter list (pPaths) as an array
        //of null-terminated strings where the last item is a zero-length
        //string (i.e. just #0). We'll now iterate over this list with some
        //simple pointer math.
        pIterator := pPaths;
        while (strlen(pIterator)>0) do begin
 
          CounterPath := pIterator;
          pIterator := pIterator + Length(pIterator) + 1;
 
          //Find the counter instance ID (the part in parentheses)
          i := Pos('(', CounterPath);
          InstanceId := Copy(CounterPath, i+1, Pos(')', CounterPath)-i-1);
          //Skip the counter if it indicates the overall CPU usage,
          //we only want the per-core values.
          if InstanceId = '_Total' then continue;
 
          //Add the counter to the query
          status := PdhAddCounter(Query, PWideChar(CounterPath), 0, h);
 
          if status = ERROR_SUCCESS then begin
 
            //Expand the internal counter array if necessary
            if cnt > Length(Counters)-1 then
              SetLength(Counters, Length(Counters)+16);
 
            //Save the counter data to the array
            with Counters[cnt] do begin
              Path := CounterPath;
              Handle := h;
 
              //Create a label for this core/CPU
              mLabel := TLabel.Create(Self);
              mLabel.Parent := Self;
              mLabel.Caption := 'Core '+InstanceId;
              mLabel.Top := (mLabel.Height + 8) * cnt + 10;
              mLabel.Left := 10;
              mLabel.AutoSize := false;
              mLabel.Width := 80;
 
              //Create a "progress bar" to show the core/CPU usage
              mGauge := TGauge.Create(Self);
              mGauge.Parent := Self;
              mGauge.Top := mLabel.Top;
              mGauge.Left := mLabel.Width + 20;
              mGauge.Height := mLabel.Height+2;
              mGauge.Width := Self.ClientWidth - mGauge.Left - 10;
              mGauge.ForeColor := $0031D329;
              mGauge.Anchors := [akRight, akTop];
 
            end;
 
            inc(cnt);
          end;
        end;//while
 
        //Truncate the array to the actual number of discovered cores  
        SetLength(Counters, cnt);
 
        //Collect the first data sample
        PdhCollectQueryData(Query);
 
      end;
    end;
 
  end;
 
  //Did we get any valid counters?
  if Length(Counters) = 0 then begin
 
    //Nope. Show an unhelpful error message...
    MessageBox(Handle, 'Initialization was unsuccessful. The application will now exit.',
      'Error', MB_OK or MB_ICONEXCLAMATION);
 
    //...and quit.
    Application.Terminate;
 
  end else begin
    //Everything went well.
    //Resize the form to make sure all progress bars are visible.
    Self.ClientHeight :=
      Counters[Length(Counters)-1].mLabel.Top +
      Counters[Length(Counters)-1].mLabel.Height +
      Counters[0].mLabel.Top;
  end;
 
end;
 
procedure TForm1.Timer1Timer(Sender: TObject);
var
 i:integer;
 counterType: PDword;
 pValue:_PDH_FMT_COUNTERVALUE;
 status : cardinal;
begin
  if Query = 0 then exit;
 
  //Collect a data sample.
  PdhCollectQueryData(Query);
 
  //Iterate over all counters and update progress bars.
  for i := 0 to Length(Counters) - 1 do begin
    counterType := nil;
 
    //Get the current core/CPU usage
    status := PdhGetFormattedCounterValue(
      Counters[i].Handle, //Counter handle as returned by PdhAddCounter.
      PDH_FMT_DOUBLE,     //Get the counter value as a double-precision float.
      CounterType,        //Counter type; unused.
      pValue);            //Output buffer for the counter value.
 
    //Update the progress bar.
    if status = ERROR_SUCCESS then
      Counters[i].mGauge.Progress := Round(pValue.doubleValue);
  end;
end;
 
procedure TForm1.FormDestroy(Sender: TObject);
begin
 //Close the query when quitting.
 if Query <> 0 then PdhCloseQuery(Query);
end;
 
end.

Download

• Example source code (5 KB)
  This is a Delphi 2009 project. To compile it, you will also need to install the Jedi API library that contains PDH API headers for Delphi.
• Compiled example application (570 KB)

P.S.
Rumor has it that you may need administrative rights to use the performance counter API.

The .MDF file extension usually designates a proprietary disk image format used by Alcohol 120%. Like ISO files, MDF files contain an exact copy a CD or DVD. This format is typically used for backing up discs and also for sharing the “backups” over BitTorrent or other P2P networks.

There are several applications that can open .MDF files :

• Alcohol 120% can mount .mdf files in a virtual CD/DVD drive. Commercial, free trial available.
• MagicISO can open or mount the files. Commercial, free trial available. Note : the trial version is limited to opening 300Mb files, so I don’t recommend using this one.
• Daemon Tools can also mount .mdf files using a virtual drive. Commercial, but the Lite edition trial version has no functionality or time limitations.
• IsoBuster can also open .mdf files. Commercial, has a free trial version with no time limit.

Gah, so all the tools are commercial. Which one should you use? Personally, I would recommend IsoBuster – it’s pretty convenient and won’t nag you incessantly about upgrading to the Pro version. It can also convert a .mdf/.mds file to the ISO format which is handy if you want to use the disk image with other CD/DVD software.

How To Open a .MDF File With IsoBuster

1. Download IsoBuster & install it.
   Note : You’ll probably want to uncheck the “Include the IsoBuster toolbar” option when installing IsoBuster. It’s probably not spyware, but there’s no point in polluting your computer with random applications. Just saying.
2. Start up IsoBuster.
   (The installer usually adds a QuickLaunch icon that you can use to run the program.)
3. Open the .MDF or .MDS file by clicking on File -> Open Image File.
   It doesn’t matter whether you select the .mdf or the .mds file in the Open.. dialog. IsoBuster will automatically load the appropriate disk image anyway. Alternatively, you can open a MDF file by dragging it into the left panel of the application window (it’s the box that contains the text “No media present” when you’ve just launched the program).
4. Extract the files you need.
   To extract a file or a folder, right-click it and select Extract [file name] from the pop-up menu. You can also drag files from the IsoBuster window and drop them into any folder on your computer to copy them there.

Additional Tip : How To Convert .MDF to .ISO

This is not a widely advertised feature, but I’ve noticed that you can actually use IsoBuster to convert .MDF/.MDS files to ISO. First, open the .mdf file as described above. Then right-click on the “CD” or “DVD” entry on the left-side navigation panel and select Extract CD/DVD <Image> -> RAW (*.bin, *.iso). Choose where to save the converted file, make sure “Save as type” is set to “*.iso”, and hit Save. After a short conversion process you’ll get an .ISO file (+ a .CUE file) that you can use with any CD/DVD burning or emulation software.

Thanks for reading

As with most AI algorithms, there are relatively few PHP classes/libraries available that deal with automated text classification. In this post I’ll review the classifier scripts I managed to find.

Note : All of these scripts use some form of the naive Bayes classifier. This algorithm is easy to implement and usually yields pretty good results, but it’s worth knowing that other techniques exist.

PHP Naive Bayesian Filter

This is a simple text classifier backed by a MySQL database. You can train it to recognize any number of categories, and it will also allow you to “untrain” the classifier if you mistakenly assign the wrong category to a document. The downside is that most of the documentation is in French, but that shouldn’t be a huge problem – the code itself is easy enough to understand and commented in English. See also : an English translation of the project’s page.

b8

b8 is an actively developed spam filter implemented in PHP. This is a good choice if you need a document classifier specifically for the purpose of catching forum/comment spammers. However, with a bit of modification you could probably coerce it to do other kinds of classification.

The filter can be trained/untrained and supports several database backends – BerkleyDB (very fast), MySQL (ubiquitous, but slower when used this way) and SQLite (a file-based database). The whole thing has a fairly professional overall look and good documentation.

PHP Text Classifier

I felt I had to include this class for completeness, but I personally wouldn’t use it for anything but the simplest projects. The good news is that this classifier supports an arbitrary number of categories and has a very simple implementation (the GPL headers probably take up more space than the code itself). The bad news is that there’s almost no documentation, the source comments are in whatever language they speak in Indonesia, and storing word frequency data as a serialized object isn’t very scalable.

Roll Your Own

If the above scripts don’t suit your needs you can always write your own. For example, the Wikipedia pages on Bayesian spam filtering and naive Bayes classifiers have a good (if a somewhat math-heavy) overview a simple classification algorithm and a list of useful resources and tutorials. Good luck