The way it works is simple : when you see a web page you’d like to keep track of (e.g. an interesting forum discussion), click a button to save the current version of that page. Then, when you revisit that page at a later time, simply click another button to instantly highlight all the new text and images that have been added since your previous visit. You can also configure the script to do one or both of those steps automatically for all pages of a specific site.

Screenshots

Highlighting a new comment

Highlighting new images

The highlighting algorithm doesn't work that great on complex or frequently updated sites, yet.

Newslight's Greasemonkey menu

Download

Install Newslight.user.js (15 KB, requires FF 3.5+)

Once installed, you can access the script’ s features by right-clicking the Greasemonkey icon on your status bar and selecting “User Script Commands…” from the pop-up menu :

• Remember this version of the page – stores the current version of the page for later reference.
• Highlight changes – compares the contents of the current page to the latest saved version and highlights any new text/images. Clicking this again will toggle the highlighting on/off.
• Automatically remember pages on example.com (on/off) – automatically remember the contents of every page that you visit on this site.
• Automatically highlight changes on example.com (on/off) – automatically highlight new items on all pages of this site.

You can also use these two bookmarklets to quickly access the script’s main functions without going through the Greasemonkey menu. Just drag them to your bookmark bar :

• Remember this page
• Highlight changes

How It Works

I’ll be the first to admit that the highlighting algorithm is very primitive. For example, changing a single word in a paragraph will make the script highlight the entire paragraph, whereas swapping two paragraphs of an article will go completely undetected and un-highlighted. The reason it happens is the way the userscript works : instead of using complex tree manipulation or advanced stuff like revision tracking, it grabs all the text and image nodes from the web page and dumps them into a big lookup table (removing duplicates in the process). Then, when you ask it to highlight changes, it just loads up the aforementioned table and checks the current version of the page against it. Anything that’s not in the table gets highlighted. The actual algorithm for finding new items takes only a couple dozen lines.

For those interested in the technical details, here’s a quick run-down :

• The lookup table is just a simple wrapper around a plain JS object being used as an associative array.
• The table is encoded in JSON and stored in a localStorage database (available in FF 3.5 and up). I’m using the PersistJS library to handle the particulars, which should make it easier to port the userscript to other browsers eventually.
• I tried various ways of fetching all text nodes from a DOM tree, but using the “//text()” XPath expression turned out to be the fastest and most straightforward approach, at least in this case.
• Miscellaneous stuff, like adding/removing the CSS classes and showing pop-up notifications, is done either via direct DOM manipulation or jQuery.

There’s certainly a lot of room for improvement – e.g. configurable highlight colours, converting the script to a proper FF addon,  better UI etc. – but the inherent simplicity of the algorithm is quite intentional. It’s not meant to precisely pinpoint new items on a site, but rather to serve as a useful hint as to where you might find them; an imperfect but handy filter.

Why do it this way? Because determining what counts as new is ultimately an AI-hard problem – you’d need software with near-human intelligence to solve it. Sure, you could probably get a decent approximation by devoting a few weeks to research, but I’d rather have something up and running now, so I can collect some feedback and see if it’s worth doing that research

Big Words, or The Theoretical Part

Fluid dynamics is a complex topic. If you so much as look it up in Wikipedia, you will be immediately assaulted by integrals, differentials and other agents of mathematical insanity. To accurately model a fluid you would need a quite an in-depth understanding of physics and calculus. And while that’s certainly not something one couldn’t figure out given enough time, “proper” fluid simulation is still extremely rare in non-academic applications and games because it’s slow.

One way to create a faster (though less accurate) simulation is by using cellular automata to represent the water. A cellular automaton is basically a grid where each cell can be in one of a finite number of states, plus a set of rules that determine how a cell can change from one state to another. The rules are typically local – that is, they consider only the current cell and it’s direct neighbours when determining the new state. During each step of the simulation, you simply loop through the entire grid and apply the rules to each cell.

When simulating water or other fluids it can more appropriate to store a continuous value (e.g. water mass) in each cell, instead of using discrete states.

CA-Based Water Simulation

Before we go into details, lets take a look at the promised online demo. It will give you a good idea about what you can achieve with this approach :

(klicken Sie auf das Bild; Java required)

Alright, on to the code stuff. Lets try simulating the most common fluid – water.

The basic data structure

//Map dimensions
final int map_width = 16;
final int map_height = 16;
 
//Block types
final int AIR = 0;
final int GROUND = 1;
final int WATER = 2;
 
//Data structures
int[][] blocks = new int[map_width+2][map_height+2];
 
float[][] mass = new float[map_width+2][map_height+2],
          new_mass = new float[map_width+2][map_height+2];

I use two two-dimensional arrays to represent the simulation world. The blocks array defines a basic “map” where each cell can contain ground (solid, stops water flow), air (an empty cell, water can flow in) or water. Each water cell also has a corresponding entry in the mass array that defines how much water it contains. All water cells/blocks start out with one unit of water.

There’s also a third array – new_mass that is used to store intermediate mass values when running the simulation. Operating directly on the mass array is unadvisable because you would get various sideffects like water spreading at different speeds depending on the order in which you update the cells.

Overall algorithm

The main idea is to treat water as a slightly compressible liquid, as described in this article. In terms of implementation, this means that if there are two or more water cells stacked vertically, the bottom cells will be able to hold slightly more water than normal. This way we don’t need to explicitly track pressure to make the water level equalize in communicating vessels – we can just look at how much excess water a cell has, and move it upwards if required.

Lets set up the exact fluid properties as constants :

//Water properties
final float MaxMass = 1.0; //The normal, un-pressurized mass of a full water cell
final float MaxCompress = 0.02; //How much excess water a cell can store, compared to the cell above it
final float MinMass = 0.0001;  //Ignore cells that are almost dry

For example, if one cell contains 1.0 units of water, the cell below it should contain up to 1.02 units, the cell below that one should contain 1.04, the next one 1.06, and so on. There is one however special case that we need to consider to get a plausible simulation – what if the if the top cell contains less than MaxMass units of water? In my implementation the bottom cell will contain a proportionally smaller excess amount, not the usual mass_of_the_cell_above + MaxCompression units.

Based on the above, we can simulate water movement by looping over the mass array and applying these cellular automaton rules to each cell :

1. Take the mass of the current cell and the cell below it and figure out how much water the bottom cell should contain. If it has less than that, remove the corresponding amount from the current cell and add it to the bottom cell.
2. Check the cell to the left of this one. If it has less water, move over enough water to make both cells contain the same amount.
3. Do the same thing for the right neighbour.
4. Do the same thing as in step 1., but for the cell above the current one.

While doing this you also need to keep track of how much water the current cell still has remaining, or you could end up with negative-mass water blocks.

After the new mass of each cell has been calculated, you can safely copy over the new values to the mass array. You also need to update the blocks array so that cells that are now dry get marked as empty and vice versa.

Here’s the simulation code :

void simulate_compression(){
  float Flow = 0;
  float remaining_mass;
 
  //Calculate and apply flow for each block
  for (int x = 1; x <= map_width; x++){
     for(int y = 1; y <= map_height; y++){
       //Skip inert ground blocks
       if ( blocks[x][y] == GROUND) continue;
 
       //Custom push-only flow
       Flow = 0;
       remaining_mass = mass[x][y];
       if ( remaining_mass <= 0 ) continue;
 
       //The block below this one
       if ( (blocks[x][y-1] != GROUND) ){
         Flow = get_stable_state_b( remaining_mass + mass[x][y-1] ) - mass[x][y-1];
         if ( Flow > MinFlow ){
           Flow *= 0.5; //leads to smoother flow
         }
         Flow = constrain( Flow, 0, min(MaxSpeed, remaining_mass) );
 
         new_mass[x][y] -= Flow;
         new_mass[x][y-1] += Flow;   
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Left
       if ( blocks[x-1][y] != GROUND ){
         //Equalize the amount of water in this block and it's neighbour
         Flow = (mass[x][y] - mass[x-1][y])/4;
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain(Flow, 0, remaining_mass);
 
         new_mass[x][y] -= Flow;
         new_mass[x-1][y] += Flow;
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Right
       if ( blocks[x+1][y] != GROUND ){
         //Equalize the amount of water in this block and it's neighbour
         Flow = (mass[x][y] - mass[x+1][y])/4;
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain(Flow, 0, remaining_mass);
 
         new_mass[x][y] -= Flow;
         new_mass[x+1][y] += Flow;
         remaining_mass -= Flow;
       }
 
       if ( remaining_mass <= 0 ) continue;
 
       //Up. Only compressed water flows upwards.
       if ( blocks[x][y+1] != GROUND ){
         Flow = remaining_mass - get_stable_state_b( remaining_mass + mass[x][y+1] );
         if ( Flow > MinFlow ){ Flow *= 0.5; }
         Flow = constrain( Flow, 0, min(MaxSpeed, remaining_mass) );
 
         new_mass[x][y] -= Flow;
         new_mass[x][y+1] += Flow;   
         remaining_mass -= Flow;
       }
 
 
     }
  }
 
  //Copy the new mass values to the mass array
  for (int x = 0; x < map_width + 2; x++){
    for (int y = 0; y < map_height + 2; y++){
      mass[x][y] = new_mass[x][y];
    }
  }
 
  for (int x = 1; x <= map_width; x++){
     for(int y = 1; y <= map_height; y++){
       //Skip ground blocks
       if(blocks[x][y] == GROUND) continue;
       //Flag/unflag water blocks
       if (mass[x][y] > MinMass){
         blocks[x][y] = WATER;
       } else {
         blocks[x][y] = AIR;
       }
     }
  }
 
  //Remove any water that has left the map
  for (int x =0; x < map_width+2; x++){
    mass[x][0] = 0;
    mass[x][map_height+1] = 0;
  }
  for (int y = 1; y < map_height+1; y++){
    mass[0][y] = 0;
    mass[map_width+1][y] = 0;
  }
 
}

And here’s the function that calculates how to distribute a given amount of water between two vertically adjacent cells :

//Returns the amount of water that should be in the bottom cell.
float get_stable_state_b ( float total_mass ){
  if ( total_mass <= 1 ){
    return 1;
  } else if ( total_mass < 2*MaxMass + MaxCompress ){
    return (MaxMass*MaxMass + total_mass*MaxCompress)/(MaxMass + MaxCompress);
  } else {
    return (total_mass + MaxCompress)/2;
  }
}

Full source code

• Download the source (23 KB)
• You will need the Java-based Processing programming language to run the code.

I didn’t discuss how to render the water, or how to handle user interaction, but the source code also includes a simplistic implementation of those things.

Conclusion

CA-based algorithms have both advantages and disadvantages. On the one hand, they’re relatively fast and the simulation quality is often good enough for games and software toys. On the other, they can produce behaviour that is very unrealistic – for example, water falling into a basin will form a little “hill” and spread out slowly, instead of near-instantly as real water would. Whether this is an acceptable downside depends on your application.

For a different take on simulating fluid dynamics with cellular automata, check out this interview with the author of Dwarf Fortress.

This gave me the idea to write a PHP script that can generate a wordpress.org-compatible changelog from the revision log. It works for any plugin hosted in the official plugins directory and can be a handy starting point for adding a changelog to plugins that don’t have it yet :

• Run the changelog generator
• Download the source code (97 KB, requires PHP 5)

For those too lazy to click one of the above links, here’s a screenshot :

And I really do need to stop making these small hacks

But if we’re wasting time we should at least waste it efficiently, right? One of the inefficiencies of this game is that the serverlist is not sortable – if you want to find a populated server you have to look over the entire listing that typically contains 50-100 active server at any given moment.

That’s why I made a GreaseMonkey script that lets you sort the Minecraft serverlist as you like. It adds two new links to the serverlist – “Sort by players” and “Sort by name”. The default is to sort by the number of active players which will put the most populated servers at the top of the page (this can be easily changed by editing one line in the script).

Download userscript!

P.S.
I’m aware that blog updates have been lacking lately. I hope to remedy with this unfortunate situation in the nearest future.

Basically, this is a partial port of the popular Firefox-only script AutoPagerize. I had to leave out some of the original’s features (like automatic updates) that were impossible to implement due to Opera’s limitations, but the main functionality is still there and the modified userscript supports all the sites that AutoPagerize does.

Download

The script is available in two versions – “Full” and “Mini”. They only differ in the number of sites they support out of the box.

• Full : AutoPagerizeOpera-full.user.js (370+ KB)
  This is a dynamically generated file that includes all the site configurations listed in the AutoPagerize database at the time when you download it.
• Mini :AutoPagerizeOpera.user.js (22 KB)
  This version only supports Google Search by default. Useful if you want a smaller and slightly faster version that you can later extend to support only the sites you need.

Installation

If you haven’t used user-scripts in Opera before, here’s how to set it up :

1. Create a new folder (anywhere). Name it something like “Opera User Scripts”.
2. Download the script file and place it in that folder.
3. In Opera, go to Tools -> Preferences -> Advanced -> Content -> JavaScript options and select the folder created in step #1.
4. Click “OK”. The script will be automatically executed the next time you navigate to a new page (you don’t need to restart Opera).

Features

• Multiple colors : Each word of the search query is higlighted in a different color, for up to five user-configurable colors. The colors will “wrap around” if there are more than five words in the query, i.e. the sixth keyword will have the same color as the first, and so on.
• Phrase highlighting : The plugin understands more advanced search queries like “wordpress plugin” screenshot and will treat a quoted phrase as a single search term. See the screenshots below for an example.
• Browser compatibility : Tested and works in FF 3.0.8, Opera 9.64, Chrome 2.0.173.1 and IE 8. The settings screen is also usable in all of the above, but it works slightly better in FF/Opera.
• Simple configuration : Just a few boxes and a neat color picker from Eyecon

Screenshots

The admin interface :

The color picker :

To demonstrate how it all looks in action, I searched my test blog for “text classification” php “machine learning” AI spam. Here’s the result :

Download

highlight-search-terms.zip (58 Kb)
Extract the archive and upload it to /wp-content/plugins, preserving directory structure.

Installation

In addition to uploading and activating the plugin, you will also need to tell it where you want the search terms to be highlighted. Go to Settings -> Highlight Search Terms and enter the HTML tag ID of the area that contains the search results. You can find the ID either by examining the HTML source code of your search page or by using a webdesign-oriented Firefox extension like DOM Inspector.

In case you have no idea what the above paragraph means, you could just enter “body” (sans quotes) in the “Content ID” field. This setting will make the plugin highlight the search terms anywhere on the search page, including the sidebar and footer.

Still, the Web really is huge, and in addition to all the well-designed and accessible sites you use every day (right?), there are also some that make your eyes hurt. The vast majority of webpages are of course somewhere in between the two extremes of “perfection” and “brain damage” – some could use a better colour scheme, others lack intuitive navigation, and some are just too long or cluttered for your tastes. However, with a few client-side tweaks the experience can be much improved.

Here’s a list of Firefox extensions that let you view the Web your way.

1. Default FullZoom level

Default FullZoom level is a simple extension that lets you set a custom zoom level on a per-site basis. This can be very helpful if you some of your favourite sites look tiny and unreadable on a high-res display. You can choose whether to zoom all page elements or only text. The extension also displays the current zoom ratio in the status bar and includes a “Fit To Window” function.

2. Remove It Permanently

RIP lets you permanently hide content from webpages. Say there’s an annoying opt-in box, a distracting announcement rendered in huge red letters, or something else your trusty AdBlock can’t catch – Remove it Permanently will get rid of it. You only need to right-click the offending object and select “Remove this permanently” from the popup menu. You can also use the advanced configuration editor to specify the removed elements using XPath.

There’s also an alternative version of RIP that’s called Yet Another Remove It Permanently. YARIP introduces a few additional features like filtering content via regular expressions and white-listing. It’s not as stable as RIP, but some users prefer the YARIP’s interface.

3. Stylish

If zooming pages and hiding unwanted content isn’t enough flexibility for you, take a look at the Stylish extension. Basically, Stylish is like Greasemonkey for CSS – it gives you the ability to define user stylesheets and extensively customise the looks of any website. In addition to that, you can even tweak the Firefox UI itself with a custom stylesheet. And don’t worry if you don’t know much about CSS – you can find a lot of user-created styles in the userstyles.org catalogue.

4. HeadingsMap

HeadingsMap generates an index of a web page using the heading elements (H1, H2, H3 and so on). HeadingsMap displays this index in a sidebar and enables you to quickly navigate to any portion of the document. The extension might also be of use to web designers as it will also display headings that have no contents and highlight heading elements that skip a level in the heading structure. Oh, and it even works with frame-based pages.

5. GreatSummary

GreatSummary can create a short summary of any page. You only need to right-click anywhere on the page, click “Summarize this page” and select the number of sentences you want returned. The addon then opens a new tab containing the summary.

While this is a neat feature in theory, the summariser can yield less than perfect results in practice. For one, the generated summary comprises simply a bunch of sentences (extracted from the original text) that the algorithm judged as sufficiently “important”, without much internal coherence. Also, sometimes the summariser can get confused and include various non-content elements (i.e. menu titles and the like) in the summary.

6. Readability

Readability is a very clever script that can process a webpage and present it in a more readable format. It will hide any distracting page elements like ads and menus and apply a custom stylesheet to make the actual content look clean and crisp. You can choose between several styles (Newspaper, Novel, etc) and configure text size and margin width.

Oh, and I cheated a bit with this post’s headline – Readability isn’t an addon. It’s actually a bookmarklet, so it will work in most modern browsers and not just Firefox

7. AutoPager

Of course, this list would be woefully incomplete without AutoPager. This addon will automatically load the next page of content (AJAX-style) when you reach the end of the current page, completely eliminating the need to constantly click “Next” to navigate a paged site. It works with most popular sites right out of the box and you can always add a custom configuration for new sites if need be.

If you have an unstable program that you absolutely need to run at all times, but don’t want to waste your time monitoring and manually restarting it every time it croaks, I might have something interesting for you.

Restart on Crash is an monitoring tool that will watch the applications that you specify and automatically relaunch any program that hangs or crashes. You can add any number of applications to monitor, enable/disable them individually and edit the command line that will be used to restart an application.

Restart on Crash doesn’t require installation and stores all it’s configuration data in a “settings.ini” file in the program’s folder, so it’s portable. It should be compatible with most NT-based Windows versions.

Download Restart on Crash (606 Kb)

Screenshots & Documentation

The main window

• To add a new application to monitor, click the “Add” button or press the Ins key.
• To delete on or more applications from the list, select them and click “Delete” or press Del.
• To edit the per-application configuration, double-click the correspoding row. This will open the editing dialog (see below).
• You can also access the RoC configuration by clicking “Settings” and view the activity log by clicking “Show Log”. The log contains information about crashed/hanged applications, executed commands, and so on.

Editing the monitor settings for an application

The configuration dialog. Yes, that's it.

“Grace period” is how long Restart on Crash will wait before trying to terminate/restart an application that it has just terminated/restarted. This is intended to prevent a scenario where RoC kills a hanged program, restarts it, decides it has hanged again (e.g. if the program is non-responsive while starting up) and wrongfully terminates it again.

Known Issues

• If you configure RoC to automatically kill a hanged application, it will terminate all instances of that application when doing so. This may be fixed eventually.
• When autorun is enabled, RoC will not start minimized or hidden in the system tray – instead, it will launch as it normally would. I originally wanted to make it load in the systray, but apparently one of the component libraries I use harbors a hard-to-find bug that makes the application behave very strangely when loaded like that.
• Overall, this is an initial release – there are bound to be other bugs. Use at your risk

Now imagine you are a typical Internet user. You’ve got a bit of technical background, so you know that your ISP and some other companies (e.g. Google) could easily track your browsing habbits – if they wanted to. Imagine you knew for certain that at least one anonymous person was actually tracking what pages you visit. How would you feel?

Anti-Privacy is a little social experiment that gives you the opportunity to explore both of these perspectives.

In more practical terms, it’s a Greasemonkey script that automatically twitters every URL you visit. You install the script, insert your Twitter credentials into the source code (it’s beta, remember? ), and your browsing history becomes public from then on. The script ignores HTTPS sites and localhost by default, but any other page is tweeted as soon as it loads. Welcome to the (semi)transparent society, netizen.

You can see an actual example feed here : http://twitter.com/antiprivacy

Installation

Firefox
You will need the Greasemonkey addon to use this script. Once you’ve got Greasemonkey up and running, right-click the link below and select “Install userscript”.

To configure the userscript, go to Tools -> Greasemonkey -> Manage User Scripts…, select the “Anti-Privacy” script and click the Edit button. Then change the twitter_username and twitter_password variables to your Twitter credentials.

Opera
I wanted to make an Opera-compatible version because Opera is my favorite browser. However, it’s userscripting capabilities are very limited, so the script needs a server-side proxy to use TinyURL and post Twitter updates.

The download below includes both the userscript and the PHP proxy script. You should upload the PHP file to your own site and change the .js file to use the new URL. You can also use the proxy script I’ve installed on this server (the default setting), but be warned that I may take it down without warning.

For installation instructions see Using user Javascript in Opera. You’ll also need to edit the userscript and insert your Twitter username/password as indicated in the file.

Disclaimer

This software is the polar opposite of secure. It’s also buggy. No sane human should use it. Ever. Well, unless you really want to.

Image source : Theron Trowbridge @ flickr

For example, here’s one I made for my Last.fm account :

Make your own!

In case you’re wondering, the mashup uses the LyricWiki API to retrieve the lyrics and a homegrown tag cloud algorithm to visualize them. When calculating the size of each word/tag, it counts how frequently that word occurs in all of the user’s top tracks and takes into account number of times each song has been played.

The results seem impressively accurate to me. Reflect my general mood, they do.