There are some (though very few they may be) edge case where regular expressions might work better, so I will discuss both approaches in this post.

Extracting Tags With DOM

PHP 5 comes with a usable DOM API built-in that you can use to parse and manipulate (X)HTML documents. For example, here’s how you could use it to extract all link URLs from a HTML file :

//Load the HTML page
$html = file_get_contents('page.htm');
//Create a new DOM document
$dom = new DOMDocument;
 
//Parse the HTML. The @ is used to suppress any parsing errors
//that will be thrown if the $html string isn't valid XHTML.
@$dom->loadHTML($html);
 
//Get all links. You could also use any other tag name here,
//like 'img' or 'table', to extract other tags.
$links = $dom->getElementsByTagName('a');
 
//Iterate over the extracted links and display their URLs
foreach ($links as $link){
	//Extract and show the "href" attribute. 
	echo $link->getAttribute('href'), '<br>';
}

In addition to getElementsByTagName() you can also use $dom->getElementById() to find tags with a specific id. For more complex tasks, like extracting deeply nested tags, XPath is probably the way to go. For example, to find all list items with the class “foo” containing links with the class “bar” and display the link URLs :

//Load the HTML page
$html = file_get_contents('page.htm');
//Parse it. Here we use loadHTML as a static method
//to parse the HTML and create the DOM object in one go.
@$dom = DOMDocument::loadHTML($html);
 
//Init the XPath object
$xpath = new DOMXpath($dom);
 
//Query the DOM
$links = $xpath->query( '//li[contains(@class, "foo")]//a[@class = "bar"]' );
 
//Display the results as in the previous example
foreach($links as $link){
	echo $link->getAttribute('href'), '<br>';
}

For more information about DOM and XPath see these resources :

• DOM API documentation
• W3Schools XPath tutorial
• XPath language specification

Honourable mention : Simple HTML DOM Parser is a popular alternative HTML parser for PHP 5 that lets you manipulate HTML pages with jQuery-like ease. However, I personally wouldn’t recommend using it if you care about your script’s performance, as in my tests Simple HTML DOM was about 30 times slower than DOMDocument.

Extracting Tags & Attributes With Regular Expressions

There are only two advantages to processing HTML with regular expressions – availability and edge-case performance. While most parsers require PHP 5 or later, regular expressions are available pretty much anywhere. Also, they are a little bit faster than real parsers when you need to extract something from a very large document (on the order of 400 KB or more). Still, in most cases you’re better off using the PHP DOM extension or even Simple HTML DOM, not messing with convoluted regexps.

That said, here’s a PHP function that can extract any HTML tags and their attributes from a given string :

/**
 * extract_tags()
 * Extract specific HTML tags and their attributes from a string.
 *
 * You can either specify one tag, an array of tag names, or a regular expression that matches the tag name(s). 
 * If multiple tags are specified you must also set the $selfclosing parameter and it must be the same for 
 * all specified tags (so you can't extract both normal and self-closing tags in one go).
 * 
 * The function returns a numerically indexed array of extracted tags. Each entry is an associative array
 * with these keys :
 * 	tag_name	- the name of the extracted tag, e.g. "a" or "img".
 *	offset		- the numberic offset of the first character of the tag within the HTML source.
 *	contents	- the inner HTML of the tag. This is always empty for self-closing tags.
 *	attributes	- a name -> value array of the tag's attributes, or an empty array if the tag has none.
 *	full_tag	- the entire matched tag, e.g. '<a href="http://example.com">example.com</a>'. This key 
 *		          will only be present if you set $return_the_entire_tag to true.	   
 *
 * @param string $html The HTML code to search for tags.
 * @param string|array $tag The tag(s) to extract.							 
 * @param bool $selfclosing	Whether the tag is self-closing or not. Setting it to null will force the script to try and make an educated guess. 
 * @param bool $return_the_entire_tag Return the entire matched tag in 'full_tag' key of the results array.  
 * @param string $charset The character set of the HTML code. Defaults to ISO-8859-1.
 *
 * @return array An array of extracted tags, or an empty array if no matching tags were found. 
 */
function extract_tags( $html, $tag, $selfclosing = null, $return_the_entire_tag = false, $charset = 'ISO-8859-1' ){
 
	if ( is_array($tag) ){
		$tag = implode('|', $tag);
	}
 
	//If the user didn't specify if $tag is a self-closing tag we try to auto-detect it
	//by checking against a list of known self-closing tags.
	$selfclosing_tags = array( 'area', 'base', 'basefont', 'br', 'hr', 'input', 'img', 'link', 'meta', 'col', 'param' );
	if ( is_null($selfclosing) ){
		$selfclosing = in_array( $tag, $selfclosing_tags );
	}
 
	//The regexp is different for normal and self-closing tags because I can't figure out 
	//how to make a sufficiently robust unified one.
	if ( $selfclosing ){
		$tag_pattern = 
			'@<(?P<tag>'.$tag.')			# <tag
			(?P<attributes>\s[^>]+)?		# attributes, if any
			\s*/?>					# /> or just >, being lenient here 
			@xsi';
	} else {
		$tag_pattern = 
			'@<(?P<tag>'.$tag.')			# <tag
			(?P<attributes>\s[^>]+)?		# attributes, if any
			\s*>					# >
			(?P<contents>.*?)			# tag contents
			</(?P=tag)>				# the closing </tag>
			@xsi';
	}
 
	$attribute_pattern = 
		'@
		(?P<name>\w+)							# attribute name
		\s*=\s*
		(
			(?P<quote>[\"\'])(?P<value_quoted>.*?)(?P=quote)	# a quoted value
			|							# or
			(?P<value_unquoted>[^\s"\']+?)(?:\s+|$)			# an unquoted value (terminated by whitespace or EOF) 
		)
		@xsi';
 
	//Find all tags 
	if ( !preg_match_all($tag_pattern, $html, $matches, PREG_SET_ORDER | PREG_OFFSET_CAPTURE ) ){
		//Return an empty array if we didn't find anything
		return array();
	}
 
	$tags = array();
	foreach ($matches as $match){
 
		//Parse tag attributes, if any
		$attributes = array();
		if ( !empty($match['attributes'][0]) ){ 
 
			if ( preg_match_all( $attribute_pattern, $match['attributes'][0], $attribute_data, PREG_SET_ORDER ) ){
				//Turn the attribute data into a name->value array
				foreach($attribute_data as $attr){
					if( !empty($attr['value_quoted']) ){
						$value = $attr['value_quoted'];
					} else if( !empty($attr['value_unquoted']) ){
						$value = $attr['value_unquoted'];
					} else {
						$value = '';
					}
 
					//Passing the value through html_entity_decode is handy when you want
					//to extract link URLs or something like that. You might want to remove
					//or modify this call if it doesn't fit your situation.
					$value = html_entity_decode( $value, ENT_QUOTES, $charset );
 
					$attributes[$attr['name']] = $value;
				}
			}
 
		}
 
		$tag = array(
			'tag_name' => $match['tag'][0],
			'offset' => $match[0][1], 
			'contents' => !empty($match['contents'])?$match['contents'][0]:'', //empty for self-closing tags
			'attributes' => $attributes, 
		);
		if ( $return_the_entire_tag ){
			$tag['full_tag'] = $match[0][0]; 			
		}
 
		$tags[] = $tag;
	}
 
	return $tags;
}

Usage examples

Extract all links and output their URLs :

$html = file_get_contents( 'example.html' );
$nodes = extract_tags( $html, 'a' );
foreach($nodes as $link){
	echo $link['attributes']['href'] , '<br>';
}

Extract all heading tags and output their text :

$nodes = extract_tags( $html, 'h\d+', false );
foreach($nodes as $node){
	echo strip_tags($link['contents']) , '<br>';
}

Extract meta tags :

$nodes = extract_tags( $html, 'meta' );

Extract bold and italicised text fragments :

$nodes = extract_tags( $html, array('b', 'strong', 'em', 'i') );
foreach($nodes as $node){
	echo strip_tags( $node['contents'] ), '<br>';
}

The function is pretty well documented, so check the source if anything is unclear. Of course, you can also leave a comment if you have any further questions or feedback.

Side-note : No, I haven’t been completely consumed by an inexorable desire to build fun graphical toys instead of practically useful software, why do you ask? In fact, there’ll be some WordPress-related announcements later this week…

Learning To Fly Fall

While I can’t claim intimate familiarity with the algorithms the aforementioned games use to simulate fire or explosions, it’s pretty easy to come up with a way to implement falling sand/flowing water. First, you need to figure out under what conditions something can fall/flow downwards. For a basic simulation this could be just a small set of very simple rules :

• For each entity that is affected by gravity
  • If there’s an empty space below it, move it down.
  • If there’s an empty space down and to the left, move it down and to the left.
  • If there’s an empty space down and to the right, move it down and to the right.

Next, you need to decide how to represent the simulation world and all the objects (like immovable walls, moving water particles et cetera) that it contains. One of the most common solutions is to use a two-dimensional array where each array element is an numeric ID of the material contained in that map tile. For example, 0 = air (or an empty square), 1 = wall, 2 = water, and so on.

In most falling sand games each array element would correspond to a single pixel on the game’s screen. However, in my example application I used 20×20 pixel tiles instead to make the underlying simulation behaviour easier to see.

If possible, use byte or char as the array element datatype. Using int’s might seem more straightforward, but it would also take up at least four times more memory and hurt performance. It’s also a good idea to make the array just a bit bigger than the actual map dimensions and create a single-tile “buffer zone” around the map so that you can avoid checking for boundary conditions during the simulation.

Finally, during the simulation step we just loop over the entire array and apply the simple rules I mentioned above to each tile that contains material that should be affected by gravity.

Getting It Right

If you go ahead and implement the above algorithm as written, you will get a simulation that kind-of works, but quickly grinds to a halt on big maps and sometimes behaves in unexpected ways. For example, you might see sand tiles that seem to “teleport” to the bottom of the map instead of falling gradually and water that always flows left after hitting a free-standing ground tile (even though there’s a perfectly suitable channel available on the right side). Lets see what we can do about that.

Keeping track of updated tiles

The “teleportation” behaviour can happen if our simulation algorithm doesn’t keep track of which tiles it has already moved around during the current simulation step. For example, first it loops over the top row of the map, finds a water tile A and moves it downwards one row. Then it loops over the second row, finds the tile A, moves it again, goes to the third row, moves the same water tile again… and so on, until the water tile A hits a solid tile or falls off the map.

To prevent this bug we need to somehow remember which tiles have been updated already. Since I wanted to keep the memory use low, I used the most-significant bit of each map array element to indicate whether that tile was last updated on an even-numbered simulation step (0) or an odd-numbered step (1). Then, at the start of each step, the algorithm calculates what the flag should be for tiles updated in that step :

update_mask = byte ( ( simulationFrameCount & (long)1 ) << 7 );
//this basically takes the modulo of frame-count / 2 and shifts the result 7 places to the left

…then sets the msb of each tile it updates to that value and automatically skips tiles that have it already set to that value.

Pseudo-random movement

When a falling particle hits a free-standing fixed tile on a 2D map it has two choices – it can either move down and left, or down and right. In the naive algorithm I discussed above it will always try to move left first, leading to weirdness. Obviously we want it to choose at random, but on a big map generating a new random number for each tile would be slow. Instead, lets use the same approach as for the “teleportation” problem and move it either diagonally left if the current simulation step is even-numbered and or diagonally right if it’s odd-numbered. This can still introduce some artifacts in the simulation behaviour, but they’re rarer and and significantly less noticeable.

Optimization

Speaking of optimization, we don’t really need to update each gravity-enabled tile every time. Many of them will eventually end up in a situation where they can’t move anywhere (e.g. water at the bottom of a puddle or sand resting on a flat surface), so it would be nice if we could quickly skip them and not waste our time trying to apply the simulation rules to these static tiles. This would help performance quite a bit.

The way I implemented is with the help with another bit-flag. If the simulation algorithm discovers that it can’t move a tile in any direction, it will set the tile’s “static” bit-flag. It will also skip any tiles that have the “static” bit-flag set.

There’s some additional book-keeping we need to do to make this work as intended. When an empty space is created somewhere in the map (e.g. by a water tile moving down) we need to re-activate any gravity-enabled tiles that are directly above the newly created hole, so that they can fall/flow down into that space.

Possible Improvements

The result is a pretty basic and plain simulation that is useful to explain the basic principle, but it wouldn’t really qualify as a full game. Here are some ideas you could use to improve it :

• Make water flow horizontally if there’s another water tile on top of it. Consider adding a simple pressure model [link to my prev. article].
• Implement fluid density. This one is easy – if the current tile is more dense than the tile below it, swap them.
• Add tile generators, e.g. a faucet that creates a water tile below itself on each simulation step (included in my demo application).
• To improve performance, use a quadtree to keep track of which regions of the map have interacting tiles and quickly skip inactive regions.
• Explore the source code of existing games and look for interesting ideas. Some open-source examples : wxSand, SDL Sand.

Live Demo & Source

Like my previous fluid simulation app, this one was also written in Processing. In addition to a working implementation of the algorithm discussed above it also lets you draw and erase water/ground/faucet tiles using your mouse, pan around the map with the WASD keys, zoom in/out with the mouse wheel and more. However, be warned that the map rendering code is not very optimized so the simulation will degenerate into a slideshow if you zoom all the way out, regardless of how fast your system is.

• Live demo  (requires Java)
• Source code

Big Words, or The Theoretical Part

Fluid dynamics is a complex topic. If you so much as look it up in Wikipedia, you will be immediately assaulted by integrals, differentials and other agents of mathematical insanity. To accurately model a fluid you would need a quite an in-depth understanding of physics and calculus. And while that’s certainly not something one couldn’t figure out given enough time, “proper” fluid simulation is still extremely rare in non-academic applications and games because it’s slow.

One way to create a faster (though less accurate) simulation is by using cellular automata to represent the water. A cellular automaton is basically a grid where each cell can be in one of a finite number of states, plus a set of rules that determine how a cell can change from one state to another. The rules are typically local – that is, they consider only the current cell and it’s direct neighbours when determining the new state. During each step of the simulation, you simply loop through the entire grid and apply the rules to each cell.

When simulating water or other fluids it can more appropriate to store a continuous value (e.g. water mass) in each cell, instead of using discrete states.

CA-Based Water Simulation

Before we go into details, lets take a look at the promised online demo. It will give you a good idea about what you can achieve with this approach :

(klicken Sie auf das Bild; Java required)

Alright, on to the code stuff. Lets try simulating the most common fluid – water.

The basic data structure

//Map dimensions
final int map_width = 16;
final int map_height = 16;
 
//Block types
final int AIR = 0;
final int GROUND = 1;
final int WATER = 2;
 
//Data structures
int[][] blocks = new int[map_width+2][map_height+2];
 
float[][] mass = new float[map_width+2][map_height+2],
          new_mass = new float[map_width+2][map_height+2];

I use two two-dimensional arrays to represent the simulation world. The blocks array defines a basic “map” where each cell can contain ground (solid, stops water flow), air (an empty cell, water can flow in) or water. Each water cell also has a corresponding entry in the mass array that defines how much water it contains. All water cells/blocks start out with one unit of water.

There’s also a third array – new_mass that is used to store intermediate mass values when running the simulation. Operating directly on the mass array is unadvisable because you would get various sideffects like water spreading at different speeds depending on the order in which you update the cells.

Overall algorithm

The main idea is to treat water as a slightly compressible liquid, as described in this article. In terms of implementation, this means that if there are two or more water cells stacked vertically, the bottom cells will be able to hold slightly more water than normal. This way we don’t need to explicitly track pressure to make the water level equalize in communicating vessels – we can just look at how much excess water a cell has, and move it upwards if required.

Lets set up the exact fluid properties as constants :

//Water properties
final float MaxMass = 1.0; //The normal, un-pressurized mass of a full water cell
final float MaxCompress = 0.02; //How much excess water a cell can store, compared to the cell above it
final float MinMass = 0.0001;  //Ignore cells that are almost dry

For example, if one cell contains 1.0 units of water, the cell below it should contain up to 1.02 units, the cell below that one should contain 1.04, the next one 1.06, and so on. There is one however special case that we need to consider to get a plausible simulation – what if the if the top cell contains less than MaxMass units of water? In my implementation the bottom cell will contain a proportionally smaller excess amount, not the usual mass_of_the_cell_above + MaxCompression units.

Based on the above, we can simulate water movement by looping over the mass array and applying these cellular automaton rules to each cell :

1. Take the mass of the current cell and the cell below it and figure out how much water the bottom cell should contain. If it has less than that, remove the corresponding amount from the current cell and add it to the bottom cell.
2. Check the cell to the left of this one. If it has less water, move over enough water to make both cells contain the same amount.
3. Do the same thing for the right neighbour.
4. Do the same thing as in step 1., but for the cell above the current one.

While doing this you also need to keep track of how much water the current cell still has remaining, or you could end up with negative-mass water blocks.

After the new mass of each cell has been calculated, you can safely copy over the new values to the mass array. You also need to update the blocks array so that cells that are now dry get marked as empty and vice versa.

Here’s the simulation code :

void simulate_compression(){
  float Flow = 0;
  float remaining_mass;
 
  //Calculate and apply flow for each block
  for (int x = 1; x <= map_width; x++){
     for(int y = 1; y <= map_height; y++){
       //Skip inert ground blocks
       if ( blocks[x][y] == GROUND) continue;
 
       //Custom push-only flow
       Flow = 0;
       remaining_mass = mass[x][y];
       if ( remaining_mass <= 0 ) continue;
 
       //The block below this one
       if ( (blocks[x][y-1] != GROUND) ){
         Flow = get_stable_state_b( remaining_mass + mass[x][y-1] ) - mass[x][y-1];
         if ( Flow > MinFlow ){
           Flow *= 0.5; //leads to smoother flow
         }
         Flow = constrain( Flow, 0, min(MaxSpeed, remaining_mass) );
 
         new_mass[x][y] -= Flow;
         new_mass[x][y-1] += Flow;   
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Left
       if ( blocks[x-1][y] != GROUND ){
         //Equalize the amount of water in this block and it's neighbour
         Flow = (mass[x][y] - mass[x-1][y])/4;
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain(Flow, 0, remaining_mass);
 
         new_mass[x][y] -= Flow;
         new_mass[x-1][y] += Flow;
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Right
       if ( blocks[x+1][y] != GROUND ){
         //Equalize the amount of water in this block and it's neighbour
         Flow = (mass[x][y] - mass[x+1][y])/4;
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain(Flow, 0, remaining_mass);
 
         new_mass[x][y] -= Flow;
         new_mass[x+1][y] += Flow;
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Up. Only compressed water flows upwards.
       if ( blocks[x][y+1] != GROUND ){
         Flow = remaining_mass - get_stable_state_b( remaining_mass + mass[x][y+1] );
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain( Flow, 0, min(MaxSpeed, remaining_mass) );
 
         new_mass[x][y] -= Flow;
         new_mass[x][y+1] += Flow;   
         remaining_mass -= Flow;
       }
 
 
     }
  }
 
  //Copy the new mass values to the mass array
  for (int x = 0; x < map_width + 2; x++){
    for (int y = 0; y < map_height + 2; y++){
      mass[x][y] = new_mass[x][y];
    }
  }
 
  for (int x = 1; x <= map_width; x++){
     for(int y = 1; y <= map_height; y++){
       //Skip ground blocks
       if(blocks[x][y] == GROUND) continue;
       //Flag/unflag water blocks
       if (mass[x][y] > MinMass){
         blocks[x][y] = WATER;
       } else {
         blocks[x][y] = AIR;
       }
     }
  }
 
  //Remove any water that has left the map
  for (int x =0; x < map_width+2; x++){
    mass[x][0] = 0;
    mass[x][map_height+1] = 0;
  }
  for (int y = 1; y < map_height+1; y++){
    mass[0][y] = 0;
    mass[map_width+1][y] = 0;
  }
 
}

And here’s the function that calculates how to distribute a given amount of water between two vertically adjacent cells :

//Returns the amount of water that should be in the bottom cell.
float get_stable_state_b ( float total_mass ){
  if ( total_mass <= 1 ){
    return 1;
  } else if ( total_mass < 2*MaxMass + MaxCompress ){
    return (MaxMass*MaxMass + total_mass*MaxCompress)/(MaxMass + MaxCompress);
  } else {
    return (total_mass + MaxCompress)/2;
  }
}

Full source code

• Download the source (23 KB)
• You will need the Java-based Processing programming language to run the code.

I didn’t discuss how to render the water, or how to handle user interaction, but the source code also includes a simplistic implementation of those things.

Conclusion

CA-based algorithms have both advantages and disadvantages. On the one hand, they’re relatively fast and the simulation quality is often good enough for games and software toys. On the other, they can produce behaviour that is very unrealistic – for example, water falling into a basin will form a little “hill” and spread out slowly, instead of near-instantly as real water would. Whether this is an acceptable downside depends on your application.

For a different take on simulating fluid dynamics with cellular automata, check out this interview with the author of Dwarf Fortress.

This gave me the idea to write a PHP script that can generate a wordpress.org-compatible changelog from the revision log. It works for any plugin hosted in the official plugins directory and can be a handy starting point for adding a changelog to plugins that don’t have it yet :

• Run the changelog generator
• Download the source code (97 KB, requires PHP 5)

For those too lazy to click one of the above links, here’s a screenshot :

And I really do need to stop making these small hacks

Here’s a very interesting video from Google TechTalks that discusses a number of advanced JS optimization techniques that will help your site avoid that horrible fate :

Bonus : If you’re a WordPress dev and intend to use the bundled jQuery library, check out these 25 tips for improving your jQuery usage. This is one of the most comprehensive jQuery tip sheets I’ve seen.

Also : Wolfenstein 3D implemented in JavaScript

Still, readability metrics can be useful as a rough indicator of how understandable your writing is. Other possible uses include detecting smart-ass comments and even catching spam (obviously, most spam messages would score very low on a readability test).

If you want to do something like that in your own PHP application(s), take a look at PHP Text Statistics. It’s a PHP class that implements a number of readability tests, including :

• Flesch-Kincaid Reading Ease (probably the most popular metric)
• Flesch-Kincaid Grade Level
• Gunning-Fog Index
• Coleman-Liau Index
• SMOG index
• Automated Reability Index

It can also calculate other metrics like sentence count, syllable count, average words per sentence, and so on.

The class is very easy to use – just include one file, create an instance, and call one of the suggestively named methods. There’s not much “official” documentation but the source code is well-commented and easy to understand.

The only flaw that stuck out to me was how inefficient the script can be. You probably won’t notice it unless you need to analyze huge volumes of text, but it could really use some optimization. In fact, I managed to improve the performance by ~30% simply by eliminating a few redundant function calls (you can download the modified version here if you’re interested).

Ah well, it’s a useful class overall. I’m sure it will come in handy the occasional NLP exploit or mashup.

Hmm, this gives me an idea for a WordPress plugin….

Windows API includes a subset of functions that provide various performance-related information, which includes data on how busy individual processors or cores are. In this post I’ll show you how to retrieve and use this information. I’ve also included a simple example application that demonstrates the functionality (the download link is at the end of the post) :

Screenshot of the example application

The example app was written in Delphi, but the salient parts are almost purely Windows API so they should be easy to translate to other programming languages.

Algorithm Overview

Here’s a general overview of how to use the performance counter API to get the per-core usage numbers.

Initialization

1. Create a performance query using PdhOpenQuery.
2. Generate a list of performance counter paths (one for each CPU or core) by feeding a wildcard path to PdhExpandWildCardPath. In this case, this is the path that we need :
   \Processor(*/*#*)\% Processor Time
   You can read more about counter path syntax here.

   Note : Unfortunately, the API makes no distinction between multiple physical processors and multiple cores, so you will get a “…\Processor(X)\…” entry for each core.

3. Add the generated counter paths to the query using PdhAddCounter and save the returned handle(s) for later.

Collecting & Displaying Data
CPU usage is typically measured over an interval of time, so you will need to collect at least two data samples at different times to get a meaningful result. Depending on your needs, you could either :

• Collect one sample, sleep() for a second, collect another sample and calculate the result; OR
• Set up a timer or a loop to calculate & displays the CPU usage periodically. That’s what I’ll do in this tutorial.

Regardless of whether you only need a one-time measurement or periodic updates, the process is very similar API-wise. First, call PdhCollectQueryData to update all counters associated with the query. Then call PdhGetFormattedCounterValue with each counter handle to get the actual data. If you set the output format flag to PDH_FMT_DOUBLE the function will give you the CPU/core load percentage as a floating-point value in the range of [0..100].

If you need periodic updates just call these two functions repeatedly.

Cleanup
When you’re done, call PdhCloseQuery to close counters associated with the query and free up allocated system resources. You don’t need to explicitly remove individual counters.

Source

This is the source code of a simple Delphi 2009 application that displays the load percentage of each processor core in a progress bar. The output is updated every second by using a timer component.

unit CoreUsage;
 
interface
 
uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, JwaWindows, ExtCtrls, ComCtrls, Gauges;
 
type
  TForm1 = class(TForm)
    Timer1: TTimer;
    procedure FormCreate(Sender: TObject);
    procedure Timer1Timer(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;
 
  TCounter = record
   Path : string;
   Handle : cardinal;
   mLabel : TLabel;
   mGauge : TGauge;
  end;
 
var
  Form1: TForm1;
  Counters : array of TCounter;
  Query : Cardinal = 0;
 
implementation
 
{$R *.dfm}
 
procedure TForm1.FormCreate(Sender: TObject);
var
 dwSize, h : cardinal;
 cnt, i : Integer;
 pPaths:PWideChar;
 pIterator:PwideChar;
 InstanceId, CounterPath : string;
 status : PDH_STATUS;
begin
  //Create a performance query
  if  PdhOpenQuery(nil, 0, Query) = ERROR_SUCCESS then begin
 
    //Get all valid processor/core usage counter paths by expanding
    //a wildcard path. To do this, we must first call PdhExpandWildCardPath
    //with buffer size set to zero to get the actual required buffer size.
    dwSize := 0;
    pPaths := nil;
 
    status := PdhExpandWildCardPath(
      nil,                                  //search the local computer
      '\Processor(*/*#*)\% Processor Time', //we want CPU usage counters for all CPUs/cores
      pPaths,                               //user-allocated buffer; currently null
      dwSize,                               //buffer size
      0) ;                                  //no flags
 
    if status = PDH_MORE_DATA then begin
      dwSize := dwSize + 1; //+1 byte required in XP and below.
      pPaths := GetMemory(dwSize); //Allocate an output buffer.
 
      //Really get the counter paths.
      status := PdhExpandWildCardPath(
        nil,
        '\Processor(*/*#*)\% Processor Time',
        pPaths,
        dwSize,
        0) ;
 
      if status = ERROR_SUCCESS then begin
 
        cnt := 0;
        SetLength(Counters, 32);
 
        //PdhExpandWildCardPath returns the counter list (pPaths) as an array
        //of null-terminated strings where the last item is a zero-length
        //string (i.e. just #0). We'll now iterate over this list with some
        //simple pointer math.
        pIterator := pPaths;
        while (strlen(pIterator)>0) do begin
 
          CounterPath := pIterator;
          pIterator := pIterator + Length(pIterator) + 1;
 
          //Find the counter instance ID (the part in parentheses)
          i := Pos('(', CounterPath);
          InstanceId := Copy(CounterPath, i+1, Pos(')', CounterPath)-i-1);
          //Skip the counter if it indicates the overall CPU usage,
          //we only want the per-core values.
          if InstanceId = '_Total' then continue;
 
          //Add the counter to the query
          status := PdhAddCounter(Query, PWideChar(CounterPath), 0, h);
 
          if status = ERROR_SUCCESS then begin
 
            //Expand the internal counter array if necessary
            if cnt > Length(Counters)-1 then
              SetLength(Counters, Length(Counters)+16);
 
            //Save the counter data to the array
            with Counters[cnt] do begin
              Path := CounterPath;
              Handle := h;
 
              //Create a label for this core/CPU
              mLabel := TLabel.Create(Self);
              mLabel.Parent := Self;
              mLabel.Caption := 'Core '+InstanceId;
              mLabel.Top := (mLabel.Height + 8) * cnt + 10;
              mLabel.Left := 10;
              mLabel.AutoSize := false;
              mLabel.Width := 80;
 
              //Create a "progress bar" to show the core/CPU usage
              mGauge := TGauge.Create(Self);
              mGauge.Parent := Self;
              mGauge.Top := mLabel.Top;
              mGauge.Left := mLabel.Width + 20;
              mGauge.Height := mLabel.Height+2;
              mGauge.Width := Self.ClientWidth - mGauge.Left - 10;
              mGauge.ForeColor := $0031D329;
              mGauge.Anchors := [akRight, akTop];
 
            end;
 
            inc(cnt);
          end;
        end;//while
 
        //Truncate the array to the actual number of discovered cores  
        SetLength(Counters, cnt);
 
        //Collect the first data sample
        PdhCollectQueryData(Query);
 
      end;
    end;
 
  end;
 
  //Did we get any valid counters?
  if Length(Counters) = 0 then begin
 
    //Nope. Show an unhelpful error message...
    MessageBox(Handle, 'Initialization was unsuccessful. The application will now exit.',
      'Error', MB_OK or MB_ICONEXCLAMATION);
 
    //...and quit.
    Application.Terminate;
 
  end else begin
    //Everything went well.
    //Resize the form to make sure all progress bars are visible.
    Self.ClientHeight :=
      Counters[Length(Counters)-1].mLabel.Top +
      Counters[Length(Counters)-1].mLabel.Height +
      Counters[0].mLabel.Top;
  end;
 
end;
 
procedure TForm1.Timer1Timer(Sender: TObject);
var
 i:integer;
 counterType: PDword;
 pValue:_PDH_FMT_COUNTERVALUE;
 status : cardinal;
begin
  if Query = 0 then exit;
 
  //Collect a data sample.
  PdhCollectQueryData(Query);
 
  //Iterate over all counters and update progress bars.
  for i := 0 to Length(Counters) - 1 do begin
    counterType := nil;
 
    //Get the current core/CPU usage
    status := PdhGetFormattedCounterValue(
      Counters[i].Handle, //Counter handle as returned by PdhAddCounter.
      PDH_FMT_DOUBLE,     //Get the counter value as a double-precision float.
      CounterType,        //Counter type; unused.
      pValue);            //Output buffer for the counter value.
 
    //Update the progress bar.
    if status = ERROR_SUCCESS then
      Counters[i].mGauge.Progress := Round(pValue.doubleValue);
  end;
end;
 
procedure TForm1.FormDestroy(Sender: TObject);
begin
 //Close the query when quitting.
 if Query <> 0 then PdhCloseQuery(Query);
end;
 
end.

Download

• Example source code (5 KB)
  This is a Delphi 2009 project. To compile it, you will also need to install the Jedi API library that contains PDH API headers for Delphi.
• Compiled example application (570 KB)

P.S.
Rumor has it that you may need administrative rights to use the performance counter API.

As with most AI algorithms, there are relatively few PHP classes/libraries available that deal with automated text classification. In this post I’ll review the classifier scripts I managed to find.

Note : All of these scripts use some form of the naive Bayes classifier. This algorithm is easy to implement and usually yields pretty good results, but it’s worth knowing that other techniques exist.

PHP Naive Bayesian Filter

This is a simple text classifier backed by a MySQL database. You can train it to recognize any number of categories, and it will also allow you to “untrain” the classifier if you mistakenly assign the wrong category to a document. The downside is that most of the documentation is in French, but that shouldn’t be a huge problem – the code itself is easy enough to understand and commented in English. See also : an English translation of the project’s page.

b8

b8 is an actively developed spam filter implemented in PHP. This is a good choice if you need a document classifier specifically for the purpose of catching forum/comment spammers. However, with a bit of modification you could probably coerce it to do other kinds of classification.

The filter can be trained/untrained and supports several database backends – BerkleyDB (very fast), MySQL (ubiquitous, but slower when used this way) and SQLite (a file-based database). The whole thing has a fairly professional overall look and good documentation.

PHP Text Classifier

I felt I had to include this class for completeness, but I personally wouldn’t use it for anything but the simplest projects. The good news is that this classifier supports an arbitrary number of categories and has a very simple implementation (the GPL headers probably take up more space than the code itself). The bad news is that there’s almost no documentation, the source comments are in whatever language they speak in Indonesia, and storing word frequency data as a serialized object isn’t very scalable.

Roll Your Own

If the above scripts don’t suit your needs you can always write your own. For example, the Wikipedia pages on Bayesian spam filtering and naive Bayes classifiers have a good (if a somewhat math-heavy) overview a simple classification algorithm and a list of useful resources and tutorials. Good luck

Of course, the list is far from comprehensive – it wasn’t really my goal to include every embedded database in existance, but rather to find those that fit the needs of the project. The focus was on speed, support for multi-threading and connectivity (what programming language interfaces are available for the DB). I also left out databases that are tied to a single language or framework (Java/.NET) – sorry about that, I know there are some good ones.

Without further ado, here are my top choices (in minimalistic list format).

#1 SQLite

Connectivity : Any programming language (almost)
Platforms : Linux, Mac OS X, OS/2, Windows (Win32 and WinCE) and embedded devices
Footprint : 300 Kb (or 180 Kb with optional features ommitted)
Model : Relational (SQL92 with some exceptions)
License : Public domain

SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain. Includes both a native C library and a simple command line client for its database.

Highlights :

#2 Firebird SQL

Connectivity : .NET, C/C++, Delphi, Java, PHP, Perl, Python, VB, COBOL, etc (see also)
Platforms : Windows (Win 98 and up), Linux (tricky)
Footprint : 4-5 MB
Model : Relational (SQL92 and most of SQL99)
License : Open source (LGPL)

Firebird is a relational database offering many ANSI SQL standard features. (…) Firebird offers excellent concurrency, high performance, and powerful language support for stored procedures and triggers. It has been used in production systems, under a variety of names, since 1981.

Highlights :

• + Referential integrity is supported, which is nice when compared to SQLite.
• + Concurrency support is also decent.
• + Supports triggers and stored procedures.
• - A notably confusing and disjointed website. I had to dig around for a while to find out what I needed to know. Some sections also present outdated information.
• - No built-in full text search.
• - If you casually browse around Friebird’s site you may get the impression that the embedded version only works with .NET (they even have an entire sub-website dedicated to the .NET version). However, you can actually use it with many other languages by downloading the embedded version and using the standard Firebird APIs.

#3 ScimoreDB

Connectivity : C++, .NET, Delphi (discontinued?)
Platforms : Windows (XP, Vista, Windows Server 2000 – Windows Server 2008)
Footprint : < 4 MB
Model : Relational
License : Free for commercial and non-commercial use

(There is no quote here because almost everything on the website reads like a shallow advertisement. Nevertheless, the feature list looked attractive enough.)

Highlights :

#4 Oracle Berkley DB

Connectivity : C, C++, Java, Tcl
Platforms : *nix, Windows, Windows CE, VxWorks, S60, BREW
Footpring : 400 Kb (minimum)
Model : Key-value
License : Open source (dual-licensed)

Berkeley DB delivers the same robust data storage features as traditional, relational database systems, such as ACID transactions and recovery; locking, multiple processes and multi-threading for high concurrency; hot and cold backup; and single-master replication for high availability applications. Berkeley DB can manage databases in memory, on disk or both.

Highlights :

Divide And Conquer

The basic idea is to split the plugin into several .php files and load only the necessary components on each particular page. The main plugin file should contain only the code that decides which other files to load (e.g. by using conditional tags). For example, if your plugin only needs to run in the Dashboard, you could place all the important code in “admin.php” and put this in the main file :

1
2
3
4
5
6
7
8
9

/*
...the plugin header...
*/
 
//Are we running in the Dashboard?
if ( is_admin() ){
    //Load the actual plugin
    require 'admin.php';
}

This way all the complex functionality – the “meat” of your plugin – won’t be needlessly loaded when someone visits the blog’s homepage or browses the archives, and the site will be a little bit faster as a result.

The Tricky Part

The above example works fine if you use the is_admin() function, but it won’t work as expected with other conditional tags like is_single and is_category – they will always return false. This is because WordPress loads plugins very early, before it has parsed the current URL and figured out what it points to.

To get around this you can hook your loader script to the “wp” action. This action executes right after the query has been parsed and posts have been loaded, so the conditional tags will return correct values by then.

The new script might look like this :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

//Are we in the Dashboard?
if ( is_admin() ){
    //Yep. Load the backend.
    require 'admin.php';
} else {
    //Nope. Set a hook to execute when other conditional tags are available.
    add_action('wp', 'load_test_plugin');
}
 
function load_test_plugin(){
    //You can use most conditional tags here.
    //For example, load the plugin only when viewing a single page or post : 
    if ( is_singular() ){
        //Load the actual plugin
        require 'core.php';
    }
}

And presto, optimization!

Caveats

• You can’t use the above method if your plugin needs to interact with one of the actions/filters that run before the wp action.
• Be aware of variable scope. When you include a file inside a function (like the hook above), any variables defined in the file will only be accessible from within that function. If you need a variable to be global, declare it explicitly with the “global” keyword.
• Another thing to keep in mind is that in this scenario the __FILE__ constant won’t always point to the main plugin file.
• It can be very hard to restructure an existing plugin to use the architecture suggested above, especially if it is very complex and needs to interact with many different hooks and filters.

Now I wish more of my 32 active plugins were optimized like this As far as I know, Ozh’s dropdown menu is the only one of them that uses this structure.